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PROTOTYPE  PROCEDURES  TO  DESCRIBE  ARMY  JOBS 


EXECUTIVE  SUMMARY 


Research  Requirement: 

Descriptions  of  Army  jobs  or  Military  Occupational  Specialties  (MOS)  provide  the 
foundation  for  Army  personnel  management,  from  entry-level  selection  and  classification  to 
training  and  performance  management.  However,  existing  job  analysis  approaches  used  in  the 
Army  have  a  number  of  limitations.  Consequently,  existing  job  analysis  approaches  make 
conducting  Army  research  to  enhance  personnel  management  both  challenging  and  costly.  The 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  convened  a  six- 
member  Army  Classification  Research  Panel  (ACRP)  in  2007  to  recommend  solutions  to  this 
issue.  The  ACRP  formulated  a  series  of  recommendations  grounded  in  occupational  or  job 
analysis  and  proposed  a  long-term  research  roadmap  for  examining  these  recommendations 
(Campbell  et  al.,  2007).  This  project  represents  the  first  step  in  that  roadmap.  The  purpose  of  this 
project  was  to  develop  and  field  test  a  new  prototype  job  analysis  approach,  customized  to  the 
Army,  for  use  in  describing  entry-level  enlisted  jobs. 

Procedure: 

Job  analysis  questionnaires  were  developed  to  measure  work  (work  activities  and  work 
context)  and  worker-oriented  domains  (work  abilities,  interests,  values,  and  styles)  useful  for 
describing  similarities  and  differences  in  entry-level  MOS.  Participants  in  the  field  test  were 
junior  enlisted  Soldiers  (E2-E4s)  and  noncommissioned  officers  (NCOs;  E5-E7s)  in  six  MOS: 

(a)  Infantryman  (1  IB),  (b)  Armor  Crewman  (19K),  (c)  Signal  Support  Specialist  (25U),  (d) 
Light- Wheel  Vehicle  Mechanic  (63B),  (e)  Military  Police  (3  IB),  and  (f)  Motor  Transport 
Operator  (88M). 

The  job  analysis  questionnaires  were  administered  online  using  the  InterForm  software 
from  ARI’s  Occupational  Analysis  (OA)  program.  To  restrict  the  administration  time  per 
participant  to  20-25  minutes  (or  less),  the  questionnaires  were  administered  using  a  matrix 
sampling  approach  (i.e.,  not  all  participants  completed  all  questionnaires).  A  total  of  four  blocks 
of  questionnaires  were  administered  to  participants. 

Completed  questionnaires  were  received  from  a  total  of  1,390  Soldiers  (1,111  supervisors 
and  277  incumbents).  The  response  rates  varied  by  MOS  and  block,  but  were  generally  less  than 
10%. 


Findings: 

Overall,  the  results  of  the  field  test  were  encouraging,  as  evidenced  by  the  following: 

The  multi-rater  reliability  estimates  were  consistently  .80  or  greater,  with  many  .90  and 
above,  indicating  that  the  mean  ratings  evidence  sufficient  reliability  to  support  their  use  in 
operational  decision-making  (e.g.,  clustering  MOS).  These  estimates  were  generally  comparable 
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to  or  exceeded  those  obtained  on  corresponding  questionnaires  administered  to  Army 
supervisors  during  an  evaluation  of  the  use  of  the  existing  0*NET  system  to  describe  Army  jobs. 
In  particular,  the  estimates  for  worker  abilities  obtained  using  the  prototype  approach  were 
consistently  higher  than  those  observed  using  the  0*NET  work  abilities  questionnaire  (a  mean 
multi-rater  ICC  of  .95  using  the  prototype  approach  versus  .75  for  the  0*NET). 

Overall,  there  was  greater  within  group  variability  than  between  group  variability, 
indicating  that  most  of  the  systematic  variance  in  ratings  resulted  from  yet-to-be  examined 
factors  (e.g.,  duty  position),  rather  than  rater  type.  Differences  between  incumbent  and 
supervisor  ratings,  on  average,  were  generally  low.  The  absolute  standardized  mean  differences 
in  ratings  between  the  two  rater  types  were,  on  average,  about  one-third  of  a  standard  deviation 
or  smaller  (mean  \d\  <  .33),  compared  to  the  standard  deviations  within  any  rater  type,  which 
were  typically  .70  or  greater.  Further,  there  was  no  evidence  that  the  differences  followed  a 
systematic  pattern  (i.e.,  one  group  consistently  rated  the  descriptors  within  a  given  domain  higher 
or  lower  than  the  other  group).  Similarly,  the  correlations  in  the  ratings  profile  between  the  two 
rater  types  were  generally  high,  on  average  (mean  r  of  .53  or  greater),  with  correlations  in  the 
.80s  for  work  activities,  evidence  that  incumbents  and  supervisors  were  generally  consistent  in 
the  relative  importance  they  assigned  the  job  descriptors  within  any  given  domain. 

Ratings  from  the  questionnaires  demonstrated  promise  for  differentiating  among  MOS, 
although  the  differences  were  small  in  several  instances.  With  the  exception  of  work  styles  and 
values,  the  mean  correlations  in  rating  profiles  across  MOS  for  any  single  domain  were 
consistently  lower,  on  average,  than  the  correlations  between  rater  types  within  an  MOS  (with 
differences  in  mean  r  ranging  from  .09  to  .22).  A  similar  pattern  of  results,  however,  was  not 
evident  when  examining  the  average  absolute  standardized  mean  differences,  where  the  mean 
differences  between  MOS  were  generally  comparable  to  or  lower  than  those  reflecting 
differences  between  rater  types  within  an  MOS  (the  differences  in  mean  \d\  ranged  from  .02  to 
.22).  Among  the  worker-oriented  questionnaires,  the  ones  demonstrating  the  greatest  potential, 
on  average,  for  differentiating  across  MOS  were  worker  abilities  and  interests,  at  least  for  this 
particular  sample  of  MOS  (Abilities:  mean  r  =  .55,  mean  \d\  =  .32;  Interests:  mean  r  =  .67;  mean 
\d\  =  .34). 

Participants  ’  reactions  to  the  questionnaires  and  their  conten  t  were  generally  positive, 
both  in  terms  of  their  ease  of  use  and  their  relevance  for  describing  their  MOS.  About  62%  or 
greater  of  the  participants,  on  average,  found  the  questionnaires  easy  to  use  and  the  instructions 
and  content  clear,  with  upwards  of  87%  of  participants  reacting  that  way  to  the  work  activities 
questionnaires.  Similarly,  62%  or  more  of  the  participants,  on  average,  rated  the  questionnaires 
as  useful  and  relevant  to  describing  their  MOS,  with  upwards  of  88%  of  participants  rating  the 
work  activities  questionnaires  in  that  manner.  Overall,  participants  reacted  more  favorably  to  the 
work-oriented  questionnaires,  in  particular  the  work  activities  questionnaire,  than  the  worker- 
oriented  questionnaires.  This  finding  is  not  surprising,  given  that  the  work-oriented 
questionnaires,  specifically  those  measuring  work  activities,  contained  the  most  job-specific 
content. 

Although  the  results  of  the  field  test  were  generally  promising,  this  research  was  not 
without  its  limitations.  Specifically: 
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The  response  rates  to  the  questionnaires  were  generally  low,  particularly  for  incumbents. 
Among  incumbents,  response  rates  were  typically  5%  or  less.  The  response  rates  among 
supervisors  tended  to  be  two  to  three  times  that  rate  (10%  to  15%),  with  a  high  close  to  31%. 
There  were  also  notable  differences  in  response  rates  by  MOS,  with  25U  generally  evidencing 
the  highest  response  rates  and  1  IB  and  19K  demonstrating  the  lowest  response  rates.  Although 
low,  these  response  rates  were  consistent  with  those  typically  encountered  by  the  Army’s 
Occupational  Analysis  Program  (OAP). 

The  ratings  collected  did  not  permit  an  examination  of  the  similarities  and  differences  in 
work  activities  by  MOS.  Although  ratings  were  collected  on  work  activities  during  the  field  test, 
these  data  were  on  the  lower-order  (or  more  job-specific)  activities.  As  a  result,  no  data  were 
available  to  evaluate  the  psychometric  properties  and  discriminant  validity  associated  with  a 
questionnaire  measuring  higher-order  (or  cross  job)  descriptors.  However,  some  evidence  for  the 
potential  value  of  such  a  measure,  albeit  indirect,  was  evident  in  the  results  for  the  work  interests 
questionnaire.  The  work  interests  questionnaire  measured  the  importance  of  interests  in 
performing  various  higher-order  work  activities  required  for  the  job.  The  content  of  the 
questionnaire  was  based  on  a  slightly  modified  version  of  the  0*NET’s  Generalized  Work 
Activities  (GWA)  taxonomy.  In  general,  the  work  interests  measure  evidenced  higher  multi-rater 
reliability  estimates,  greater  discriminant  validity,  and  more  favorable  participant  reactions  than 
the  other  worker-oriented  questionnaires. 

Although  the  evidence  for  discriminant  validity  was  promising,  the  differences  in  ratings 
by  MOS  from  any  single  domain  were  generally  small.  For  example,  the  average  absolute 
standardized  mean  differences  by  MOS  ranged  from  .18  (Work  Context  and  Work  Values)  to  .34 
(Work  Interests).  There  are  several  potential  explanations  for  this  finding:  (a)  the  nature  of  the 
MOS  sampled;  (b)  the  use  of  importance  ratings,  as  opposed  to  level  ratings  or  some  alternative 
kind  of  rating  exercise  with  greater  potential  to  differentiate  across  MOS;  (c)  the  questionnaires 
contained  a  number  of  items  not  useful  for  differentiating  across  MOS,  or  at  least  this  particular 
sample  of  MOS;  and  (d)  the  use  of  web-based  surveys  to  collect  ratings,  as  opposed  to  some 
alternative  data  collection  method  (e.g.,  a  workshop). 

Utilization  and  Dissemination  of  Findings: 

Based  on  the  results  of  the  field  test,  the  next  recommended  step  is  to  expand  it  to  a  larger 
sample  of  MOS,  as  outlined  in  the  ACRP’s  research  roadmap  (Campbell  et  al.,  2007).  In  the 
near-tenn,  collecting  ratings  on  a  larger  sample  of  MOS  would  potentially  serve  two  purposes: 

(a)  to  develop  MOS  clusters  that  could  be  used  to  select  MOS  to  target  in  future  criterion-related 
validation  research  (e.g.,  to  examine  the  classification  potential  of  new  predictor  measures);  and 

(b)  to  develop  or  to  identify  end-of-training  criteria  that  sufficiently  support  the  Anny’s 
personnel  classification  research,  but  require  fewer  resources  to  construct  and  to  field  (e.g., 
occupational  judgment  tests).  In  the  long  term,  collecting  these  ratings  would  enable  the  Army  to 
test  procedures  for  generalizing  or  transporting  criterion-related  validation  evidence  obtained  for 
a  smaller,  focal  sample  of  MOS  to  a  larger  sample  (e.g.,  to  facilitate  the  development  or  updating 
of  prediction  equations  to  be  used  in  classifying  new  Soldiers). 

For  the  purposes  of  meeting  the  near-term  objective  of  clustering  MOS,  this  expanded 
sample  of  MOS  would  need  to  be  larger  than  the  number  of  focal  MOS  to  be  sampled  in  the 
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planned  criterion-related  validation  research.  This  sample  would  preferably  comprise  20-30  jobs. 
Accordingly,  job  analysis  data  would  need  to  be  collected  from  an  additional  45-55  MOS  to 
ensure  a  sufficiently  representative  number  to  meet  this  goal.  Assuming  that  only  cross-job 
descriptors  were  used  so  none  of  the  questionnaires  would  be  MOS-specific,  this  could  be  a 
feasible  next  step. 
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Prototype  Procedures  to  Describe  Army  Jobs 


I.  Introduction  and  Project  Objectives 
Background 

Describing  Army  jobs  or  Military  Occupational  Specialties  (MOS)  provides  the 
foundation  for  Anny  personnel  management,  from  entry-level  selection  and  classification  to 
training  and  performance  management.  However,  existing  operational  approaches  to  analyze  and 
describe  Anny  MOS  have  several  limitations. 

•  They  vary  within  and  across  MOS.  These  differences  are  evident  in  the  descriptors 
used,  as  well  as  the  process  used  to  describe  them.  As  a  result,  every  MOS  “seems” 
different.  Sometimes  the  same  MOS  “seems”  different. 

•  They  sometimes  capture  information  that  is  too  detailed  or  too  specific  to  be  useful 
for  a  given  personnel  management  application.  Other  times  these  approaches  capture 
information  that  is  not  descriptive  enough.  Further,  the  level  of  detail  varies 
considerably  both  across  and  within  MOS. 

•  They  sometimes  miss  important  information  about  the  requirements  of  the  MOS  (e.g., 
non-technical  work  activities,  worker-oriented  characteristics). 

Consequently,  existing  job  analysis  approaches  make  conducting  Army  research  to 
enhance  personnel  management  both  challenging  and  costly.  The  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences  (ARI)  convened  a  six-member  Army  Classification 
Research  Panel  (ACRP)  in  2007  to  recommend  solutions  to  these  limitations;  limitations  that 
have  and  continue  to  carry  significant  implications  for  conducting  enlisted  classification  research 
(e.g.,  collecting  criterion  data  on  a  sufficiently  representative  sample  of  MOS).1 

The  ACRP  considered  a  number  and  diverse  range  of  strategies  and  solutions  to  these 
limitations,  ranging  from  occupational  or  job  analysis  to  criterion  measurement  development  to 
analytic  methods  for  generalizing  or  transporting  predictive  validity  evidence.2  They  concluded 
that  a  viable,  long-term  solution  to  these  limitations  required  a  comprehensive,  evidence-based 
approach  that  addressed  all  these  areas.  The  ACRP’s  deliberations  resulted  in  a  series  of 
recommendations  and  a  long-term  roadmap  of  research  activities  designed  to  examine  and 
inform  one  or  more  of  these  recommendations  (see  Figure  1).  The  first  step  in  that  roadmap  was 
to  develop  and  field  test  a  standardized  approach  for  describing  entry-level  jobs.  This  job 
analysis  approach  (or  system)  would  possess  certain  features  (e.g.,  cross-job  descriptors 
customized  to  the  Army  context,  work  activity  descriptors  at  multiple  levels  of  description)  and 
could  be  implemented  Army-wide.  The  ACRP  concluded  that  having  such  an  approach  would 
provide  the  necessary  foundation  for  future  research  efforts  to  advance  the  Army’s  classification 
system  from  clustering  MOS  to  developing  new  end-of-training  criteria  to  generalizing  or 
transporting  predictive  validity  estimates  across  MOS. 


1  The  results  of  the  ACRP’s  deliberations  and  their  recommendations  can  be  found  in  Campbell,  McCloy,  McPhail, 
Pearlman,  Peterson,  Rounds,  and  Ingerick  (2007). 

2  Throughout  this  report,  “occupational  analysis,”  “work  analysis,”  and  “job  analysis”  are  used  interchangeably. 
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Accordingly,  ARI  has  undertaken  two  projects  to  investigate  and  evaluate  valid 
approaches  for  describing  MOS.  The  first  of  these  projects,  Evaluating  the  0*NET  Occupational 
Analysis  System  for  Army  Competency  Developmen  t,  examined  the  utility  of  the  existing 
Occupational  Information  Network  (0*NET)  for  describing  entry-level  enlisted  and  officer  jobs 
(Russell  et  ah,  2008).  The  second  of  these  projects,  which  is  the  focus  of  this  report,  involved 
developing,  piloting,  and  field  testing  a  prototype  job  analysis  approach  for  use  in  describing 
entry-level  enlisted  MOS,  using  standardized  descriptor  taxonomies  and  questionnaires 
customized  to  the  Army. 


Project  Objectives 

The  primary  purpose  of  this  project  was  to  develop  and  field  test  a  prototype  job  analysis 
approach  for  describing  Army  jobs.  Consistent  with  the  roadmap  proposed  by  the  ACRP,  the 
specific  objectives  of  this  effort  were  twofold: 

•  To  design,  develop,  and  field  test  a  prototype  job  analysis  approach  that  (a) 
sufficiently  describes  similarities  and  differences  in  Anny  MOS,  and  (b)  can  be 
implemented  Army-wide  and  at  a  reasonable  cost. 

•  To  develop  and  start  to  populate  a  supporting  relational  database  that  would 
systematically  collect  and  store  job  analysis  data  and  other  results  (e.g.,  criterion- 
related  validity  estimates),  where  feasible.  The  database  will  provide  useful 
information  toward  meeting  the  Anny’s  personnel  management  objectives.  The  initial 
application  would  be  to  support  new  Soldier  selection  and  classification. 
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Overview  of  Report 


This  report  is  organized  as  follows.  First,  we  discuss  the  reasons  for  and  the  essential 
features  of  the  proposed  job  analysis  approach.  Second,  we  describe  development  of  the 
proposed  approach  and  the  questionnaires.  Third,  we  summarize  administration  of  the 
questionnaires  in  the  field  test.  Fourth,  we  discuss  the  results  and  findings  of  the  field  test.  The 
report  concludes  with  a  summary  of  the  major  findings  of  the  field  test  and  recommendations  for 
next  steps. 


II.  Reasons  for  and  Essential  Features  of  an 
Army-Specific  Job  Analysis  Approach 

Why  an  Alternative  Army-Specific  Approach? 

Having  job  analysis  data  represents  an  essential  first  step  in  meeting  the  Army’s  needs  for 
research  on  Soldier  classification  and  other  personnel  issues.  The  ACRP  observed  that  job 
analysis  approaches  currently  in  use  operationally  in  the  Army,  while  useful  for  their  intended 
purposes  (e.g.,  designing  MOS-specific  training  programs),  do  not  provide  the  information 
needed  for  classification  and  personnel  research  for  the  reasons  previously  cited  (e.g.,  lack  of 
standardization  across  MOS,  incomplete  or  missing  information  on  important  job  requirements). 
The  ACRP  considered  but  concluded  that  other  existing  job  analysis  systems,  such  as  the 
0*NET,  could  provide  the  basis  for  an  Army  job  analysis  system  and  that  there  were  advantages 
to  linking  Army  content  to  existing  systems.  However,  these  existing  systems  were  not  likely  to 
fully  meet  the  Army’s  needs  in  their  current  form  since  they  were  developed  to  describe  civilian, 
not  military,  jobs.  In  sum,  the  ACRP  determined  that  existing  systems  available  to  the  Army  for 
describing  entry-level  MOS: 

•  Miss  important  requirements  useful  for  describing  Army  jobs  (e.g.,  combat-related 
work  activities  or  work  context  characteristics  unique  to  the  Army  or  a  military 
setting); 

•  Include  descriptors  that  are  not  relevant  for  describing  Army  jobs  or  use  tenninology 
(or  examples)  that  does  not  accurately  capture  entry-level  Army  job  requirements; 
and 

•  Fail  to  capture  job  information  at  multiple  levels  of  description,  so  as  to  serve  the 
Army’s  need  for  information  for  multiple  purposes  (e.g.,  entry-level  selection  and 
classification,  training,  performance  measurement). 

What  Kind  of  an  Approach  Does  the  Army  Need? 

For  these  reasons,  the  ACRP  concluded  that  the  Army  should  develop  a  new  job  analysis 
system  that  functions  effectively  within  the  context  of  the  Army.  The  ACRP  recommended  that 
the  system  include  the  following  features: 

•  A  common  language,  customized  to  the  Army  context,  for  describing  similarities  and 
differences  in  MOS.  This  common  language  should  consist  of  a  reasonably 
comprehensive  set  of  descriptors  representing  targeted  work  (i.e.,  work  activities,  work  or 
job  context,  machine-tools-equipment-technology)  and  worker-oriented  (i.e.,  knowledge, 
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skills,  abilities,  work  values,  interests)  domains.  These  descriptors  must  be  sufficient  for 
describing  any  MOS. 

•  Cross-MOS  descriptors  for  use  in  identifying  similarities  and  differences  across  MOS. 

•  Descriptors,  in  particular  work  activities  (i.e.,  what  Soldiers  do  on  a  job),  at  multiple 
levels  of  generality  that  can  be  organized  hierarchically  to  support  the  Army’s  needs  for 
job  infonnation  at  multiple  levels  of  aggregation. 

•  The  flexibility  to  combine  descriptors  or  data  on  descriptors  from  multiple,  traditionally 
independent  domains,  and  in  new  or  innovative  ways  that  enable  the  identification  of 
similarities  and  differences  in  Army  jobs. 

•  A  reasonable  cost  to  implement  the  system. 

•  Sufficient  flexibility  so  that  the  descriptors  can  be  refined  and  updated,  as  needed.  The 
system  needs  to  include  a  mechanism  for  adding  new  descriptors,  or  modifying  existing 
descriptors,  as  job  information  on  a  larger  sample  of  MOS  is  collected  or  MOS 
requirements  change  over  time. 

•  A  foundation  based  on  descriptor  taxonomies,  developed  or  refined  using  a  combined 
top-down  and  bottom-up  approach,  as  recommended  by  the  ACRP. 

•  A  relational  database  that  collects,  organizes,  and  stores  job  analysis  data  systematically 
over  time  and  that  facilitates  the  integration  of  these  data  with  other  relevant  personnel 
data  (e.g.,  criterion-related  validity  estimates). 

fn  the  near-term,  having  a  job  analysis  system  with  the  above  features  will  enable  the  Army  to: 

•  Cluster  MOS  to  guide  the  sampling  of  MOS  for  criterion-related  validation  studies; 

•  Demonstrate  the  relevance  of,  or  refine,  existing  criteria  for  use  in  criterion-related 
validation  studies; 

•  Develop  new,  alternative  criteria  that  target  critical  MOS-specific  and  cross-MOS 
dimensions  useful  for  differentiating  MOS; 

•  Cluster  MOS  to  produce  experimental  job  families  that  maximize  the  (observed) 
classification  gains  of  targeted  predictors  (e.g.,  non-cognitive  predictors); 

•  Cluster  MOS  to  estimate  and  determine  the  limits  of  generalizing  (or  transporting) 
criterion-related  validity  estimates,  or  prediction  equations,  across  MOS;  and 

•  Generalize,  or  transport,  criterion-related  validity  estimates  or  performance  prediction 
equations  for  a  sample  of  MOS  to  other  MOS  sharing  the  same  requirements  for  purposes 
of  selecting  predictor  batteries  that  maximize  classification  gains. 

Over  time,  having  a  job  analysis  system  with  the  above  features  will  enable  the  Anny  to: 

•  Cluster  MOS  to  produce  new,  alternative  job  families  for  operational  use  under  the 
Army’s  current  enlisted  classification  system  that  maximize  the  classification  gains  of 
targeted  predictors; 

•  Cluster  MOS  to  investigate  potential  enhancements  to  the  operational  classification 
system  (e.g.,  a  two-stage  procedure,  whereby  recruits  are  first  assigned  to  broad  clusters 
of  MOS  on  the  basis  of  their  interests,  then  to  a  specific  MOS  within  a  cluster  based  on 
their  abilities);  and 
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•  Document  changes  in  MOS  over  time  and  their  implications  for  the  operational  use,  or 

continued  use,  of  previously  collected  criterion-related  validity  estimates  (or  prediction 

equations)  for  classifying  new  recruits  to  MOS. 

III.  Development  of  Prototype  Army  Job  Analysis  Approach 
Selection  of  Job  Descriptor  Taxonomies 

Central  to  the  proposed  job  analysis  system  are  job  descriptors  for  use  in  characterizing 
the  similarities  and  differences  in  Army  MOS.  Table  1  summarizes  the  potential  universe  of  job 
descriptor  categories  and  domains  useful  for  serving  the  Army’s  enlisted  selection  and 
classification  needs,  based  on  the  0*NET’s  Content  Model  (Peterson,  Mumford,  Borman, 
Jeanneret,  &  Fleishman,  1999)  with  representative  examples  specific  to  the  Army. 

Because  of  the  resources  available,  it  was  not  feasible  in  this  prototype  project  to 
formulate  and  collect  data  on  descriptors  encompassing  all  of  the  categories  and  domains 
referenced  in  Table  1.  As  a  result,  some  prioritization  was  needed.  Many  of  the  research 
activities  comprising  the  ACRP’s  roadmap  involve,  or  require,  clustering  MOS  on  the  basis  of 
one  or  more  targeted  descriptor  domains.  Table  2,  taken  from  the  ACRP  report  (2007;  see  pp. 
10-12),  matches  the  descriptor  domain(s)  most  useful  to  advancing  each  of  these  activities, 
ordered  in  descending  order  of  priority.  As  evident  from  the  table,  work  activities,  at  any  level  of 
description,  emerge  as  important  to  accomplishing  several  of  these  activities,  followed  by  one  or 
more  worker-oriented  characteristics  (i.e.,  occupation-specific  knowledges  and  skills,  work 
interests  or  values,  abilities).  Accordingly,  the  following  job  descriptor  domains  were  identified 
by  the  ACRP  as  most  critical  for  development  (in  descending  order  of  priority): 

•  Work  activities  (to  be  defined  at  a  minimum  of  two  levels  of  specificity) 

•  Work  (or  organizational)  context  and  related  personal  characteristics  (i.e.,  interests, 
values,  and  temperament  [or  work  styles]) 

•  Occupation-specific  knowledges  and  skills,  to  include  weapons-equipment-tools- 
technology 

•  Worker  aptitudes  and  abilities 

Guiding  Design  Principles  for  Developing  Prototype  Army  Job  Descriptors 

There  were  a  number  of  issues  to  be  addressed  in  the  development  of  the  prototype  Army 
job  descriptors.  The  implications  of  these  issues  should  not  be  understated  given  the  objectives 
of  this  initial  effort.  These  issues  included: 

•  Which  job  descriptor  domains  should  be  included?  For  each  domain  targeted,  at  what 
and  how  many  levels  of  generality  should  the  descriptors  be  specified?  At  what  level 
of  detail  should  the  descriptors  be  specified? 

•  Should  descriptors  reflecting  requirements  common  to  all  MOS  (e.g.,  common, 
Army-wide  tasks)  be  included? 

•  Should  descriptors  include  MOS-specific  content  unique  to  an  MOS  (e.g.,  MOS- 
specific  tasks,  tools  and  technology,  education  and  training  requirements)? 


5 


Table  1.  The  Potential  Universe  of  Work  and  Worker-Oriented  Descriptors  Useful  for  Serving 
the  Army’s  Enlisted  Classification  and  Selection  Needs 

Category/  Representative  Examples 

Domain _ Subdomain _ Definition _ (Specific  to  Army) _ 

Work-Oriented _ 

Work  Activities  Occupation/job-specific  The  major  duties  and  key  •  Maintains  a  motor  vehicle 

technical  work  activities  that  •  Transports  cargo  and 

constitute  the  core  personnel 

technical  requirements  of  .  Loads/unloads  passengers 
a  job  and  that  differentiate  for  transport  in  truck 
one  Army  job,  or  cluster  .  Performs  tie-down 

_ of  jobs,  from  another. _ procedures _ 

Non-occupation/job-  The  major  duties  and  key  •  Process  casualties 

specific  technical  work  activities  •  Administer  first  aid 

constituting  the  core  •  Install  and  recover  mines 

technical  requirements  •  Evade  and  survive  capture 

that  are  not  unique  to  a  job 
but  are  required  by  most 

_ or  all  jobs  in  the  Army. _ 


Non-technical 

The  major  duties  and  key 
work  activities  that  are 
non-technical,  and 
typically  intra-  and  inter¬ 
personal,  in  nature. 

•  Demonstrates  physical 
fitness 

•  Demonstrates  teamwork 

•  Exhibits  effort  and  initiative 
on  the  job 

•  Exhibits  integrity  and 
discipline  on  the  job 

•  Relates  to  and  supports 
peers 

•  Solves  problems  and  makes 
decisions 

Work  Context 

Job  context 

Characteristics  of  the 
immediate  job 
environment  (e.g., 
physical,  motivational, 
and  social)  that  influence 
the  nature  of  work 
performed. 

•  Communication  methods 

•  Job  hazards 

•  Physical  setting  and 
proximity 

•  Situational  constraints  (e.g., 
time  pressure) 

Organizational  or  unit 

Characteristics  of  the 

•  Culture/climate 

context 

organization,  or  unit 
within  the  organization, 
that  influence  the  work 
performed. 

•  (De)centralization  of 
decision-making 

•  Team-based  structure 

Weapons- 

Weapon(s),  tool(s), 

•  M16A2  rifle 

Equipment-T  ools- 

equipment,  and/or 

•  .50  caliber  machine  gun 

Technology 

technology  needed  to 
perform  the  technical  and 
non-technical 
requirements  of  a  job,  or 
cluster  of  jobs. 

•  Hoist 

•  Wrench 

•  Air  compressor 
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Table  1.  (continued) 


Category/ 

Domain 

Subdomain 

Definition 

Representative  Examples 
(Specific  to  Army) 

Work-Oriented  (continued) 

Occupational  and 

Workforce 

Characteristics 

Minimum  entry 
requirements  and  training 
difficulty 

Information  on  the 
minimum  entry 
requirements  of  job  and 
difficulty  of  its  technical 

•  Minimum  entry  requirements 
(e.g.,  Aptitude  Area  (AA)  cut 
scores) 

•  Passing  scores  for  advanced 

training. 

technical  training 
•  Completion/graduation  rates 
from  advanced  training 

Mission  importance 

Information  on  the  job’s 
importance  and  priority  to 
the  Army. 

•  Job  quotas  (or  accession 
targets) 

•  Deployment/OPTEMPO  rates 

Job  family/cluster 
membership 

Information  on  the  job’s 
existing  family  or  cluster 
membership. 

•  Aptitude  Area  (AA)  family 

•  Career  Management  Field 
(CMF) 

Internal  workforce 

Current  characteristics  of 

•  Numbers  and  demographics  of 

job  incumbents. 

current  job  incumbents 
•  Gains-promotions-losses  (in 
job) 

Worker-Oriented  • 

Personal 

Characteristics 

Occupational  or  work 
interests 

Characteristic  preferences 
for,  or  likes  and  dislikes 

•  Realistic 

•  Artistic 

regarding,  specific  work- 
related  requirements. 

•  Investigative 

•  Social 

•  Enterprising 

•  Conventional 

Occupational  or  work 
values 

Characteristic  goals, 
beliefs,  or  ideals 
regarding  specific  work- 
related  requirements. 

•  Autonomy 

•  Creativity 

•  Feedback 

•  Independence 

•  Team  orientation 

•  Variety 

Work  temperament  or 

Characteristic  behavioral 

•  Achievement/effort 

styles 

tendencies,  habits,  or 
styles  reflecting 
preferences  for  specific 
work-related 
requirements. 

•  Concern  for  others 

•  Cooperation 

•  Dependability 

•  Initiative 

•  Leadership  orientation 

•  Persistence 

•  Stress  tolerance 
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Table  1.  (continued) 


Category/ 

Domain 

Subdomain 

Definition 

Representative  Examples 
(Specific  to  Army) 

Worker-Oriented  (continued) 

Knowledges  and 
Skills 

Basic  workplace 
knowledges  and  skills 

Basic,  developed 
capacities  that  facilitate 
the  learning  and 
acquisition  of  new 
knowledges  and  skills  and 
that  are  required  to  at 
least  some  degree  in 
virtually  all  jobs. 

• 

• 

• 

Basic  math  facility 

Reading  skill  or 
comprehension 

Written  communication 

Cross-functional 

Generic,  developed 

• 

Information  gathering  and 

knowledges  and  skills 

capacities  that  facilitate 

management 

the  performance  of  the 

• 

Oral  communication 

technical  and  non¬ 

• 

Self-directed  learning  and 

technical  requirements 

development 

common  to  a  wide  range 

• 

Self-management 

of  jobs. 

• 

Teamwork 

Occupation  or  job- 

Specific,  developed 

• 

Close  combat 

specific  knowledges  and 

capacities  that  are  needed 

• 

Basic  electronic  design  and 

skills 

to  perform  the  technical 

repair 

and  non-technical 

• 

Basic  mechanical  knowledge 

requirements  specific  to 

and  repair 

an  occupation/job. 

Aptitude  and 

General 

Capacity  to  perform  the 

• 

General  cognitive  ability 

Abilities 

general  mental  and 

• 

Physical  ability 

physical  requirements  of 

• 

Psychomotor  ability 

the  job;  includes  level  of 

complexity  needed  (high. 

medium,  low). 

Specific 

Capacity  to  perform  the 

• 

Arm-hand  steadiness 

specific  mental  and 

• 

Hand-eye  coordination 

physical  requirements  of 

• 

Manual  dexterity 

the  job;  includes  level  of 

• 

Pattern  recognition 

complexity  needed  (high. 

• 

Perceptual  speed  and  accuracy 

medium,  low). 

Experience  and 

Work/job-related 

The  types  and  amounts 

• 

Athletic 

Education 

experience  and  training 

of  job/work-related 

• 

Driving  and  piloting 

experience  and  training 
desirable  or  needed  to 
perform  the  requirements 
of  the  job. 

• 

• 

• 

• 

• 

Electrical 

Health  services  and  medical 
Mechanical 

Protective  service 

Technical  vocational 

Formal  education  and 
credentials 

The  types  of  degrees 
earned  and  formal 
credentials  (e.g.,  licenses/ 
certifications)  desirable 
or  needed  to  perform  the 
requirements  of  the  job. 

• 

• 

• 

• 

Post-secondary  degree 

Graduate  degree 

Credentialing  examination 

Union,  guild,  or  professional 
association  membership 
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Table  2.  Purposes  for  Clustering  MOS  and  Their  Relevant  Descriptor  Domains 


Purpose 


Primary  Clustering 

Descriptor(s) _ Comment(s) 


1.  To  create  occupational 
clusters  to  facilitate  the 
sampling  of  MOS  and  the 
conduct  of  criterion-related 
validation  studies. 


Higher-order  and  lower-order 
work  activities;  any  or  all 
occupation-specific 
knowledges,  skills,  and 
abilities  (KSAs). 


Could  facilitate  the  conduct  of  criterion- 
related  validation  studies  by  grouping 
equivalent  occupations.  This  would  enable 
the  Army  to  obtain  a  sufficiently 
representative  sample  of  MOS  that  would 
permit  generalization  to  other  MOS,  but 
minimize  difficulties  in  obtaining  the 
requisite  data. 


2.  To  explore  and/or  detect 
candidates  for  “mid-range” 
criterion  measures. 


Higher-order  work  activities.  For  potential  use  in  studying  the  feasibility 

of  developing  new  performance  criterion 
measures,  or  refining  existing  criterion 
measures,  applicable  across  multiple  MOS. 


3.  To  research  and/or  explore  the 
utility  of  non-performance 
descriptor-based  MOS  sub¬ 
clusters,  either  in  and  of 
themselves,  or  within  broader 
performance  descriptor-based 
clusters,  for  predicting  non¬ 
performance  criteria  (i.e., 
attrition  or  retention-related).3 

4.  To  research  and/or  explore  the 
utility  of  performance 
descriptor-based  MOS  sub¬ 
clusters  within  broader  non¬ 
performance  descriptor-based 
clusters. 


Any  or  all  KSAs;  also, 
potentially,  organization/job 
context  descriptors  (for  aspects 
of  research  related  to  person- 
organization  fit  issues). 


Lower-order  and/or  higher- 
order  work  activities. 


Clustering  descriptor(s)  would  be  selected 
based  on  specific  research  question(s)  of 
interest.  Could  address  such  questions  as, 
“Are  there  clusters  of  MOS  for  which 
certain  types  or  combinations  of  KSAs  are 
predictive  of  attrition,  non-technical 
performance  (e.g.,  peer  leadership),  or 
person-organization  fit?”  b 

Could  address  such  questions  as,  “What 
MOS  within  the  existing  Aptitude  Areas 
(AA)  share  similar  performance 
requirements  (useful  for  supporting  a 
validity  or  test  transportability  approach)?”  c 


“  For  all  purposes  listed  that  involve  “researching/exploring  the  utility  of. . it  should  be  understood  the  central  question  guiding 
all  such  research  is  the  “utility  for  what?”  (i.e.,  what  is  the  criterion  on  which  to  evaluate  this  research?).  In  some  cases,  this 
criterion  could  be  the  actual  criterion  data  collected  from  the  validation  studies  (i.e.,  which  clustering  solution  maximizes 
classification  efficiency),  provided  there  are  criterion  data  for  a  sufficient  number  of  MOS.  Alternatively,  the  criterion  could  be 
more  qualitative  in  nature,  along  the  lines  of,  “Does  the  clustering  solution  answer  -  or  shed  more  light  on  -  the  question  asked?” 
These  later  cases  still  require  advance  specification  of  the  parameters  for  acceptable  answers  to  such  questions. 
b  Should  aggregate  classification  gains  accrue  from  the  use  of  alternative  criteria  (e.g.,  retention-related)  but  their  maximization 
result  in  different  MOS  assignments  than  maximizing  aggregate  technical  performance,  then  additional  research  would  be  needed 
to  scale  the  relative  value  of  gains  on  each  criterion. 

c  Such  an  approach  would  also  be  useful  for  identifying  clustering  solutions  that  maximize  technical  performance. 


•  How  should  theater  or  operation-specific  requirements  (e.g.,  Operation  Iraqi 
Freedom)  be  handled?  What  implications,  if  any,  do  they  have  for  the  “shelf-life”  of 
the  proposed  approach? 

•  How  to  incorporate  information  about  the  work  or  job  context  or  conditions  under 
which  work  must  be  performed  (e.g.,  work  activity  by  work  activity,  descriptor 
domain  by  descriptor  domain,  or  as  a  separate,  independent  descriptor  domain)? 

•  How  can  descriptors  (or  information  on  descriptors,  from  multiple,  independent 
domains)  be  combined  in  new,  innovative  ways  that  both  enhance  the  description  of 
similarities  and  differences  in  Army  jobs  and  minimize  the  costs  of  collecting  job 
data? 
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Table  2.  (continued) 


Purpose 


Primary  Clustering 

Descriptor(s)  Comment(s) 


5.  To  research  various  forms  of  Interests,  values, 
a  two-stage  classification  temperaments, 
model:  Similar  to  #1 
(above),  but  sub-cluster 
within  existing  AA  clusters/ 


A  two-stage  procedure  might  assign 
Soldiers  to  broad  clusters  first  and  then 
make  further  differential  assignments 
within  clusters.  Assignments  to  broad 
clusters  (i.e.,  the  first  stage)  could  be  on 
the  basis  of  interests  and/or  fit  with  the 
peer  leadership  and  team  support 
performance  requirements.  Specific 
assignments  (i.e.,  the  second  stage)  within 
the  broad  clusters  could  be  made  on  the 
basis  of  AFQT,  AA,  Army  priorities,  or 
openings.  Alternatively,  if  the  second 
stage  were  counseling/guidance-oriented, 
one  could  mitigate  response  bias  problems 
with  some  of  these  measures.6 


6.  To  research  various  forms  of 
a  “two-track”  (as  opposed  to 
two-stage) 

selection/classification 

model. 


It  would  depend  on  premises  of 
the  second  “track,”  but  most 
likely  applications  would 
involve  either  cross-functional 
skills  (e.g.,  oral 
communication,  teamwork, 
critical  thinking)  or 
specialized/advanced  skills  or 
knowledge,  as  might  be  used 
for  an  alternative  track  for 
experienced  applicants,  or 
applicants  with  relevant 
certifications. 


Could  address  such  questions  as,  “Are 
there  clusters  of  MOS  for  which  certain 
specialized  predictor  types  (e.g.,  cross¬ 
functional  skills,  or  prior  work 
experience)  might  be  leveraged?” 


d  The  Army’s  current  classification  process  consists  of  a  single  stage.  Specifically,  new  Army  recruits  are  shown  one  or  more 
possible  MOS  opportunities  on  a  career  counselor’s  display  screen,  then  make  a  choice.  Which  MOS  opportunities  are  presented 
is  based  on  a  joint  function  of  recruits’  AFQT  and  AA  scores,  Army  priorities,  and  technical  training  seat  availability. 

6  Because  of  the  way  the  Army’s  basic  and  technical  training  are  currently  structured,  it  would  be  difficult  to  do  multi-stage 
classification  at  different  points  in  time  (like  the  Air  Force)  by  delaying  the  final  job  assignment  until  after  basic  training. 
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•  What  metrics,  or  scales,  should  be  used  to  assess  the  targeted  descriptors?  How  will 
these  assessments  be  scored  (e.g.,  in  terms  of  importance,  needed  at  entry,  proficiency 
level  required)? 

Answers  to  all  of  these  issues  could  not  be  fully  specified  up  front  or  examined  empirically 
during  the  course  of  this  initial  prototyping  effort.  On  other  issues,  such  as  the  first  one,  preliminary 
decisions  were  made  by  the  project  team,  based  on  the  ACRP’s  recommendations.  Those  decisions 
formed  the  basis  for  a  set  of  design  principles,  which  are  summarized  in  Table  3.  These  principles 
guided  and  infonned  the  development  of  the  prototype  job  descriptors. 

Table  3.  Guiding  Design  Principles  for  Developing  Army  Job  Descriptors 

1.  The  number  of  job  descriptors  constituting  each  targeted  domain  should  be  as  comprehensive  as  possible 
(i.e.,  no  important  descriptors  should  be  missing),  but  manageable.  The  “optimal”  number  of  descriptors 
within  a  domain  should  be  sufficient  for  describing  important  similarities  and  differences  in  Army  jobs 
without  compromising  comprehensiveness  and  the  approach’s  feasibility. 

2.  Job  descriptors  must  fit  within,  and  reflect,  the  applicable  domain  definition;  for  example,  a  work 
activities  descriptor  should  reflect  a  work  activity,  as  operationalized  by  the  project  team. 

3.  The  formulation  and  selection  of  job  descriptors  should  be  grounded  in,  and  informed  by,  relevant 
theory,  existing  taxonomies,  and  past  research. 

4.  Job  descriptors,  particularly  work  activities,  should  be  specified  at  multiple  levels  of  generality  that  can 
be  organized  hierarchically  to  support  the  Army’s  needs  for  job  information  at  multiple  levels  of 
aggregation.  The  “optimal”  number  of  levels  should  be  sufficient  to  capture  both  the  specificity  needed 
for  criterion  measure  development  and  the  generality  needed  for  transporting  validity  estimates,  without 
being  too  cumbersome  to  develop  or  collect  data  on. 

5.  Job  descriptors  within  the  same  domain  and  at  the  same  level  of  generality  should  be  of  similar  breadth 
and/or  scope. 

6.  Cross-job  descriptors  -  that  is,  descriptors  that  can  be  applied  across  Army  jobs  -  should  be  defined  so 
that  a  particular  requirement  can  describe  multiple  jobs  if  the  jobs  do  in  fact  share  a  similar  requirement, 
at  some  level  of  generality. 

7.  Where  appropriate,  explore  job  descriptors,  or  information  on  descriptors,  from  multiple,  traditionally 
independent  domains  that  could  be  combined  in  new  and  innovative  ways  that  both  enhance  the  description  of 
similarities  and  differences  in  Army  jobs  and  minimize  the  costs  of  collecting  job  information. 

8.  Job  descriptor  statements  should  be  written  in  a  manner  that  reflects  their  meaning  or  application  within 
the  Army  context;  that  is,  descriptors  should  be  customized  to  the  Army  context. 

9.  Job  descriptors  within  the  same  domain  should  be  defined  to  be  as  conceptually  independent  as  possible; 
that  is,  redundancies  and  overlap  among  descriptors  representing  the  same  domain  should  be  minimized. 

10.  Job  descriptor  labels  and  statements  should  be  selected  or  developed  in  a  manner  that  maximizes  their 
definitional  integrity  and  likely  measurability;  that  is,  they  should  lend  themselves  to  the  development  of 
metrics  (or  scales)  that  permit  the  reliable  and  valid  assessment  of  Army  jobs  against  said  descriptors. 

1 1 .  Job  descriptor  labels  and  definitions  should  be  written  in  “user-friendly”  language,  without  the  use  of 
jargon  or  complex  terminology. 

12.  Job  descriptor  labels  and  definitions  should  be  written  in  a  conceptually  simple  and  clear  manner. 

13.  Job  descriptor  definitions  should  be  descriptive,  not  prescriptive  or  evaluative. 

Note.  Several  of  these  guidelines  are  adapted  from  “Recommendations  for  an  NSSB  Common  Language  for  Skill 
Standards”  (NSSB,  1998). 

Our  overall  approach  in  developing  the  descriptors  was  to  build  on  existing  job  analysis 
systems,  in  particular  the  0*NET,  and  earlier  job  analysis  efforts  for  the  Army  (e.g.,  Knapp  & 
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Campbell,  2006;  Knapp,  Sager,  &  Tremble,  2005),  where  advisable.  In  the  case  of  existing  job  analysis 
systems,  this  typically  involved  adapting  the  most  promising  descriptors  from  those  systems  by 
adding  new  ones  or  modifying  existing  descriptors  to  ensure  their  relevance  and  applicability  to  the 
Anny.  This  approach  was  in  keeping  with  the  ACRP’s  recommendation  that  the  prototype  job 
analysis  system  need  not  be  built  from  scratch,  and  that  selected  elements  of  existing  job  analysis 
systems,  with  the  appropriate  modifications,  could  prove  useful  in  describing  Army  jobs.  A  further 
advantage  to  this  approach  is  that  incorporating  elements  of  an  existing  system,  such  as  the  0*NET, 
into  the  Army  prototype  pennits  linkages  to  those  systems  and  any  relevant  data  collected  on  them 
(and  vice  versa).  These  linkages  could  be  of  value  to  the  Army  in  the  future  for  other  kinds  of 
applications  (e.g.,  transitioning  existing  Soldiers  to  civilian  jobs,  identifying  civilian  career 
development  opportunities  for  Reserve  or  National  Guard  Soldiers  to  enhance  their  skill  sets). 

Development  of  the  Prototype  Descriptors 

Based  on  the  guiding  design  principles,  a  descriptor  development  blueprint  was  created. 
Similar  to  a  test  blueprint  or  measurement  plan,  the  descriptor  blueprint  defined  each  domain  and 
specified  how  many  levels  and  at  what  level  of  detail  the  descriptors  within  that  domain  were  to  be 
written  and  the  steps  to  follow  in  developing  them.  See  Appendix  A  for  relevant  portions  of  the 
blueprint. 

Prototype  descriptors  and  accompanying  questionnaires  were  developed  for  a  total  of  six  out 
of  the  seven  domains  identified  by  the  ACRP  for  development  in  this  prototype.  Only  occupation- 
specific  knowledges  and  skills  did  not  have  descriptors  developed  for  it.  A  preliminary  effort  was 
made  to  develop  descriptors  in  this  domain,  but  the  resulting  descriptors  tended  to  closely  correspond 
to  those  being  developed  for  the  job-specific  work  activity  domain  (e.g.,  knowledge  or  skill  to  do  X). 
Accordingly,  the  development  of  occupation-specific  knowledge  and  skill  descriptors  were  dropped 
from  the  development  plan.3  Consistent  with  the  ACRP’s  recommendations,  work  activity 
descriptors  were  developed  at  multiple  levels  of  specificity.  The  highest  level  comprises  a 
standardized  set  of  cross-job  descriptors  that  could  be  used  to  consistently  rate  all  jobs  in  the  Anny, 
followed  by  two  lower  levels  of  descriptors  that  were  more  job-specific.  Ratings  data  were  not 
collected  on  these  higher-order  work  descriptors  during  the  field  test.  This  was  because  practical 
constraints  required  us  to  prioritize  the  domains  to  be  tested.  The  cross-job  work  activity  descriptors 
were  detennined  to  be  lower  priority,  relative  to  other  descriptor  domains,  because  (a)  many  of  the 
descriptors  on  which  these  cross-job  descriptors  were  based  were  first  developed  and  examined  in 
previous  Anny  funded  research  whose  goals  and  purpose  were  comparable  to  those  of  the  cunent 
effort;  and  (b)  the  limited  number  and  nature  of  the  MOS  sampled  for  the  cunent  field  test  were 
likely  to  limit  the  kinds  of  conclusions  that  could  be  drawn  about  the  descriptors’  usefulness. 
Although  the  higher-order  work  descriptors  developed  for  the  project  were  not  field  tested,  they  were 
piloted  during  the  workshops  conducted  for  the  0*NET  Evaluation  study  to  confinn  their  relevance 
for  describing  MOS  and  their  interpretability  to  Anny  supervisors  or  incumbents.  Table  4 
summarizes  the  targeted  domains  for  which  descriptors  and  questionnaires  were  developed  and  field 
tested  in  this  project,  organized  by  category  and  level  of  description  (i.e.,  cross-job  versus  job- 
specific). 


3  Consistent  with  one  of  the  ACRP’s  recommendations,  there  could  be  considerable  value  in  incorporating 
occupation-specific  knowledges  and  skill-type  descriptors  into  work  activities,  rather  than  treating  them  as  two 
separate  domains.  Specifically,  occupation-specific  knowledges  and  skills  could  be  used  to  describe  work  activities 
that  are  primarily  knowledge-based  or  cognitive  in  nature  (e.g.,  problem  solving  or  decision-making). 
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Table  4.  Descriptor  Domains  Developed  for  Prototype  Questionnaires  by  Category  and  Level 
of  Description 


Category 

Level  of  Description 

Worker-Oriented 

Work-Oriented 

Cross-Job 

•  Work  Abilities 

•  Work-Organizational  Context 

•  Work  Interests 

•  Work  Values 

•  Work  Styles 

•  Work  Activities  (Higher-Order) 

Job-Specific 

•  Work  Activities  (Lower-Order) 

In  general,  developing  the  descriptors  for  each  domain  involved  the  following  steps: 

1.  Collect  source  materials  (e.g.,  Soldier  Training  Publications)  or  compile  candidate 
descriptors  for  the  targeted  domain  from  relevant  theory  and  existing  taxonomies, 
such  as  those  available  in  the  0*NET  system. 

2.  Draft  a  preliminary  taxonomy  of  descriptors  for  the  targeted  domain  using  those 
source  materials  from  the  Army,  and  following  the  specifications  outlined  in  the 
blueprint.  Project  staff  modified  existing  taxonomies  to  make  them  relevant  to  the 
Army. 

3.  Review  the  preliminary  taxonomy  internally  for  consistency  with  the  blueprint’s 
specifications. 

4.  Conduct  an  external  review  of  the  descriptors  for  the  targeted  domain  with  military 
personnel  researchers,  military  job  analysts,  or  Army  SMEs  to  ensure  that  the 
descriptors  were  relevant,  sufficiently  comprehensive,  and  understandable. 

Table  5  summarizes  the  source(s)  that  served  as  the  primary  basis  on  which  prototype 
descriptors  for  the  targeted  domains  were  developed. 
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Table  5.  Primary  Source(s)  for  the  Descriptor  Domains  Developed  for  Prototype 


Questionnaires 

Domain 

Source(s) 

Work-Oriented 

Work  Activities 

For  the  cross-job  (or  highest  order)  work  activities,  the  Job  Activities  and 

Task  Categories  from  the  U.S.  Army  SYNVAL  project  (Peterson  et  al., 

1990).  The  specifications  for  the  job-specific  (or  lower-order)  activities  were 
modelled  after  the  performance  categories  and  requirements  from  the  U.S. 

Army  PerformM21  project  (Knapp  &  Campbell,  2006)  and  the  0*NET’s 
Detailed  Work  Activities  (DWAs)  (Dietrich,  Hendrickson-Larson,  Hoppe, 
Paige,  &  Rosenow,  2002;  National  0*NET  Center,  2003). 

Work-Organizational  Context 

0*NET  Work  Context  taxonomy  (McPhail  et  al.,  1995) 

Worker-Oriented 

Work  Abilities 

0*NET  Abilities  taxonomy  (Fleishman,  Wetrogan,  Uhlman,  &  Marshall- 
Mies,  1995) 

Work  Interests 

0*NET  Generalized  Work  Activities  (GWA)  taxonomy  (Jeanneret  & 

Borman,  1995),  supplemented  by  Holland’s  (1997)  RIASEC  (Realistic, 
Investigative,  Artistic,  Social,  Enterprising,  and  Conventional)  model,  a 
typology  for  categorizing  individuals  and  work  environments 

Work  Values 

Work  Values  taxonomy  from  the  U.S.  Army  Select21  project  (Knapp  et  al., 
2005),  which  was  partly  based  on  Dawis  and  Lofquist’s  (1984)  theory  of 
work  adjustment 

Work  Styles 

Work  Styles  taxonomy  from  the  U.S.  Army  Select21  project  (Knapp  et  al., 
2005),  which  represented  a  slightly  modified  version  of  the  0*NET  Work 

Styles  taxonomy  (Borman,  McKee,  &  Schneider,  1995) 

Selection  of  Rating  Scales 

To  capture  data  on  the  various  work-  and  worker-oriented  domain  descriptors,  we 
selected  among  several  rating  scales  used  in  previous  research.  For  all  domains  except  work 
context,  we  selected  a  5 -point  absolute  (as  opposed  to  relative)  importance  scale,  where  1  =  Not 
important  and  5  =  Extremely  important.  Theoretically,  a  relative  scale  (e.g.,  Less  important  than 
other  activities/abilities/work  styles)  might  result  in  less  negative  skew  -  which  is  often  a 
problem  with  job  analysis  ratings.  However,  relative  ratings  can  limit  comparability  across  jobs. 
Therefore,  all  the  rating  scales  used  in  the  field  test  questionnaires  used  absolute  rating  scales. 

Work  context  descriptors  were  rated  on  a  single  5-point  scale  ranging  from  1  (Not  at  all 
characteristic)  to  5  ( Very  characteristic).  Because  work  context  descriptors  (e.g.,  exposure  to 
danger)  could  be  expected  to  vary  in  combat  versus  garrison  settings,  respondents  were  asked  to 
rate  the  descriptors  twice  -  once  for  combat  and  once  for  non-combat  settings. 

In  addition  to  the  importance  rating  scale,  the  work  activities  (and  work  activity 
categories)  were  rated  on  two  additional  scales.  To  obtain  information  relevant  to  training 
applications  and  for  facilitating  the  selection  or  classification  of  Soldiers  who  will  be  successful 
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in  Initial  Entry  Training,  one  scale  rated  the  difficulty  for  first-term  Soldiers  in  their  MOS  to 
learn  to  perfonn  the  activity  through  classroom  training  (1  =  Very  easy  to  learn,  5  =  Very 
difficult  to  learn).  The  third  and  final  scale  assessed  criticality,  which  is  importance  ratings  but 
with  the  emphasis  on  the  unit’s  mission.  Specifically,  the  scale  asked  respondents  to  rate  the 
seriousness  of  the  negative  consequences  for  a  Soldier’s  unit  or  the  unit’s  mission  if  a  first-term 
Soldier  in  their  MOS  failed  to  successfully  perfonn  the  activity  ( 1  =  Not  serious,  5  =  Extremely 
serious). 

Finally,  the  worker-oriented  questionnaires  instructed  respondents  to  rank  order  the 
descriptors  within  a  targeted  domain  on  their  importance  (e.g.,  from  1  =  Most  Important  to  30  = 
Least  Important).  This  exercise  was  included  on  the  questionnaires  for  three  purposes:  (a)  to 
provide  additional  data  to  examine  participants’  responses  to  the  ratings  scales  (e.g.,  did 
participants’  importance  ratings  match  their  rankings?);  (b)  to  enhances  the  questionnaires’ 
ability  to  capture  cross-MOS  differences;  and  (c)  to  open  up  relative-based  options  for  scoring 
the  questionnaires  that  were  otherwise  not  available  using  absolute  scales.  A  similar  rank  order 
exercise  had  been  used  in  job-side  measures  developed  to  support  the  scoring  of  new  predictor 
measures  for  the  Select21  research  program  (Knapp  et  ah,  2005). 

Development  and  Pilot  Testing  of  the  Questionnaires 

After  the  descriptor  taxonomies  were  finalized,  questionnaires  were  drafted  to  measure 
the  targeted  domains.  Preliminary  versions  of  the  job-specific  work  activity  questionnaires  were 
administered  to  Army  SMEs  in  several  targeted  MOS  as  part  of  a  companion  evaluation  of  the 
use  of  the  0*NET  system  for  describing  Army  jobs  (Russell  et  ah,  2008).  All  of  the 
questionnaires  were  pilot  tested  with  Army  SMEs,  both  junior  enlisted  and  NCOs,  in  August 
2008  at  Forts  Bragg  and  Riley.  Two  significant  changes  were  made  to  the  questionnaires  on  the 
basis  of  this  pilot.  First,  the  work  abilities  taxonomy  was  condensed  to  make  the  questionnaire 
easier  to  complete  without  losing  important  infonnation.  Specifically,  the  initial  abilities 
taxonomy,  which  was  a  slightly  modified  version  of  Fleishman’s  abilities  taxonomy,  was 
condensed  from  54  abilities  to  30.  This  was  accomplished  by  merging  two  or  more  similar 
abilities  into  a  single  ability,  based  on  pilot  test  results  combined  with  rational  judgment.  Second, 
the  ranking  task  was  dropped  because  it  proved  too  cumbersome. 

Development  of  the  Supporting  Relational  Database 

At  present,  no  single  database  exists  for  centrally  storing  job  analysis  data  collected  from 
current  efforts  to  describe  Army  jobs.  Accordingly,  the  second  major  deliverable  in  this 
prototype  effort  was  a  supporting  relational  database.  As  envisioned  by  the  ACRP,  the  primary 
purpose  of  this  relational  database  is  to  systematically  capture  and  store  job  analysis  data  in  an 
ongoing  fashion.  These  job  analysis  data,  either  individually  or  in  combination  with  other 
relevant  data  (e.g.,  criterion-related  validity  estimates,  individual  level  Soldier  predictor-criterion 
data),  could  then  be  used  to  generate  and  refine  solutions  to  the  Army’s  classification  needs 
successively  over  time,  as  more  data  become  available.  Such  an  incremental  approach  balances 
demands  on  Army  resources  while  providing  the  Army  with  reasonably  sound  and  viable 
solutions  to  its  enlisted  classification  objectives. 
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At  present,  the  relational  database  contains  tables  and  fields  for  capturing  and  storing 
information  on  (a)  the  MOS  sampled,  (b)  the  prototype  job  descriptors,  (c)  the  scales  used  in  the 
questionnaires,  and  (d)  relevant  summary  statistics  on  the  ratings  collected  on  the  prototype 
descriptors  (e.g.,  means,  standard  deviations,  median,  inter-rater  reliability  estimates).  A  layout 
describing  the  contents  of  the  final  database  can  be  found  in  Appendix  B.  In  brief,  developing  the 
supporting  relational  database  involved  the  following  steps: 

1 .  Design  a  relational  database  to  encompass  the  elements  comprising  the  prototype 
approach.  After  consultations  with  the  Army,  it  was  determined  that  a  Microsoft 
Access  database  to  be  housed  on  an  Army-owned  server  would  sufficiently  serve  the 
Army’s  research  needs  at  this  early  stage. 

2.  Program  a  working  version  of  the  relational  database  in  Microsoft  Access. 

3.  Test  and  validate  the  working  database  using  data  collected  from  the  field  test. 

4.  Finalize  the  database  and  upload  it  to  an  Army-owned  server. 

The  final  database  was  delivered  to  ARI  at  the  completion  of  the  project  and  uploaded  to 
an  Army-owned  server.  Included  in  the  final  database  are  the  summary  statistics  on  the  ratings 
collected  during  the  field  test  for  the  six  MOS  sampled. 
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IV.  Field  Test  of  the  Prototype  Army  Job  Analysis  Approach 


Method 


Sample 

Participants  in  the  field  test  included  junior  enlisted  Soldiers  (E2-E4s)  and  NCOs  (E5- 
E7s)  in  six  targeted  MOS:  (a)  Infantryman  (1  IB),  (b)  Armor  Crewman  (19K),  (c)  Signal  Support 
Specialist  (25U),  (d)  Light- Wheel  Vehicle  Mechanic  (63B)4 5,  (e)  Military  Police  (3  IB),  and  (f) 
Motor  Transport  Operator  (88M).  Participants  for  each  MOS  were  randomly  selected  from 
rosters  provided  by  the  U.S.  Army’s  Human  Resource  Command  (HRC).  Both  incumbents  and 
supervisors  were  sampled  to  examine  if  the  two  groups  systematically  perceive  and,  thereby  rate, 
the  same  job  differently. 

The  goal  was  to  obtain  complete  data  from  at  least  50-60  respondents  per  questionnaire 
for  each  MOS.  We  assumed  no  greater  than  a  10%  response  rate.  When  sampling  respondents 
our  target  was  a  60-40%  split  between  junior  enlisted  Soldiers  (incumbents)  and  NCOs 
(supervisors)  within  each  MOSC  A  60-40  split  was  chosen  because  NCOs  tend  to  exhibit  higher 
response  rates,  on  average,  to  these  kinds  of  questionnaires  than  more  junior  Soldiers.  For  most 
MOS,  it  was  not  possible  to  achieve  our  60-40  targeted  split  for  various  reasons  (e.g.,  the  size  of 
the  MOS,  the  number  of  Soldiers  currently  deployed  and  thereby  inaccessible  to  participate  in 
the  field  test). 

Description  of  Questionnaires 

Work  Activities.  The  Work  Activities  questionnaires  asked  incumbents  and  supervisors  to 
rate  various  work  activities  and  work  activity  categories  specific  to  their  MOS  on  three  5-point 
scales:  (a)  importance  to  the  job  of  first-term  Soldiers  in  their  MOS,  (b)  difficulty  for  first-term 
Soldiers  in  their  MOS  to  learn  to  perform  the  activity  through  classroom  training,  and  (c) 
seriousness  of  the  negative  consequences  for  a  Soldier’s  unit  or  the  unit’s  mission  if  a  first-term 
Soldier  in  their  MOS  failed  to  successfully  perform  the  activity.  Consistent  with  the  ACRP 
recommendations  to  have  multiple  levels  of  specificity  in  task  domain,  respondents  were  asked 
to  provide  ratings  for  both  discrete  work  activities  (e.g.,  “Mount  and  dismount  machine  guns  or 
other  personal  weapons  on  tripods”)  and  categories  of  work  activities  (e.g.,  “Operate  and 
maintain  firearms  and  personal  weapons  [e.g.,  grenade  launchers,  grenades]”).  The  number  of 
work  activities  and  work  activity  categories  varied  by  MOS.  Copies  of  the  Work  Activities 
questionnaires  for  all  six  MOS  are  provided  in  Appendix  C.  Unlike  work  context,  we  did  not 
collect  two  sets  of  ratings  on  work  activities,  one  set  targeting  a  combat  environment  and  the 
second  focused  on  a  non-combat  environment.  A  questionnaire  measuring  the  cross-job  (or 
highest-order)  work  activity  descriptors  was  not  developed  and  administered  during  the  field  test. 
As  mentioned  previously,  the  cross-job  work  activity  descriptors  were  detennined  to  be  lower 
priority,  relative  to  other  descriptor  domains,  because  (a)  many  of  the  descriptors  on  which  these 
cross-job  descriptors  were  based  were  first  developed  and  examined  in  previous  Army  funded 


4  The  MOS  designation  for  this  MOS  is  changing  from  63B  to  91B.  This  report  uses  the  63B  designation. 

5  Throughout  the  remainder  of  the  report,  the  term  “incumbents”  is  used  to  refer  to  the  junior  enlisted  Soldiers  and 
“supervisors”  when  referring  to  NCOs. 
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research  whose  goals  and  purpose  were  comparable  to  those  of  the  current  effort;  and  (b)  the  limited 
number  and  nature  of  the  MOS  sampled  for  the  current  field  test  were  likely  to  limit  the  kinds  of 
conclusions  that  could  be  drawn  about  the  descriptors’  usefulness. 

Work  Context  The  Work  Context  questionnaire  (shown  in  Appendix  D)  asked 
respondents  to  rate  the  extent  to  which  46  work  context  descriptors  are  characteristic  for  first- 
term  Soldiers  in  their  MOS,  using  a  5-point  scale.  An  example  work  context  descriptor  is, 
“Exposed,  or  potentially  exposed,  to  manufactured  (i.e.,  man-made)  hazards  or  contaminants.” 
Respondents  rated  each  statement  twice,  once  for  non-combat  zone  field  assignments  and  once 
for  combat  zone  field  assignments. 

Work  Abilities.  The  Work  Abilities  questionnaire  asked  respondents  to  rate  the 
importance  of  30  ability  descriptors  (e.g.,  “Exert  maximum  muscle  force  to  lift,  push,  pull  or 
carry  objects”)  to  the  job  of  first-term  Soldiers  in  their  MOS  on  a  5-point  scale.  The  Work 
Abilities  questionnaire,  along  with  the  remaining  worker-oriented  questionnaires,  is  shown  in 
Appendix  E. 

Work  Styles.  On  the  Work  Styles  questionnaire,  respondents  rated  the  importance  of  20 
work  style  descriptors  to  the  job  of  first  tenn  Soldiers  in  their  MOS.  A  sample  work  style 
descriptor  is,  “Showing  a  cooperative  and  friendly  attitude  towards  others  they  dislike  or 
disagree  with.” 

Work  Interests.  The  Work  Interests  questionnaire  asked  incumbents  and  supervisors  to 
rate  the  importance  of  30  interest  descriptors  (e.g.,  “Protecting  others”)  to  the  job  of  first  term 
Soldiers  in  their  MOS. 

Work  Values.  The  Work  Values  questionnaire  also  requested  a  single  importance  rating 
for  each  of  30  work  values  descriptors  (e.g.,  “Plan  their  work  with  little  supervision”). 

Procedure  for  Administering  the  Questionnaires 

The  questionnaires  were  administered  online  using  ARI’s  InterForm  software  application. 
Because  our  goal  was  to  keep  the  administration  time  per  participant  to  no  more  than  20-25 
minutes,  the  questionnaires  were  administered  using  a  matrix  sampling  approach  (i.e.,  not  all 
participants  completed  all  questionnaires).  Four  blocks  of  questionnaires  were  administered  to 
participants,  as  summarized  in  Table  6. 

Table  6.  Questionnaires  Administered  in  Field  Test  by  Block 

Block _ Questionnaires  (in  Order  of  Administration) _ 

1  (a)  Work  Activities;  (b)  Work  Interests;  and  (c)  Work  Values 

2  (a)  Work  Activities;  (b)  Work  Abilities;  and  (c)  Work  Styles 

3  (a)  Work  Context;  (b)  Work  Interests;  and  (c)  Work  Values 

4  _ (a)  Work  Context;  (b)  Work  Abilities;  and  (c)  Work  Styles _ 


The  blocks  were  constructed  so  that  each  questionnaire  was  administered  in  two  of  the  four 
blocks.  Each  block  consisted  of  three  questionnaires,  the  first  measuring  one  of  the  work- 
oriented  domains  (work  activities  or  context)  and  the  other  two  targeting  the  worker-oriented 
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domains  (work  abilities,  styles,  values,  or  interests).  Participants  within  an  MOS  were  randomly 
assigned  to  each  block. 

Invitations  were  e-mailed  to  selected  participants’  Army  Knowledge  Online  (AKO) 
accounts.  The  invitation  explained  that  ARI  had  been  tasked  to  develop  a  new  job  analysis 
approach  to  support  the  Army’s  needs  of  ensuring  that  the  right  Soldier  is  classified  to  the  right 
MOS  and  that  they  were  being  asked  to  complete  a  series  of  questionnaires  to  evaluate  this  new 
approach.  The  invitation  also  stated  that  all  responses  would  be  treated  as  confidential  and  used 
for  research  purposes  only. 

The  e-mail  included  a  signed  memo  from  the  ARI  Unit  Chief  of  the  Personnel 
Assessment  Research  Unit  (PARU),  who  endorsed  the  research  project.  The  memo  was  intended 
to  increase  the  response  rate  by  signaling  the  importance  of  the  research  effort. 

The  invitations  contained  a  web  link  to  the  questionnaires.  All  of  the  questionnaires  were 
administered  on  a  secure  .mil  server.  When  accessing  the  questionnaires,  all  participants  were 
asked  to  create  a  user  ID  and  password.  The  ID  and  password  allowed  participants  to  save  their 
responses  and  exit  the  questionnaires,  so  that  they  could  return  to  them  where  they  had 
previously  left  off,  as  opposed  to  re-starting  at  the  beginning. 

Participants  were  given  a  suspense  date  of  2  weeks  after  receipt  of  the  invitation. 
Reminder  notices  were  sent  to  participants  one  week  into  the  2-week  window. 

Questionnaire  Instructions 

Consistent  with  the  Army’s  personnel  management  objectives,  participants  were 
instructed  to  rate  Skill  Level  1  duty  positions  (i.e.,  entry-level  jobs)  within  their  MOS. 
Specifically,  participants  were  asked  to  consider  “the  typical  first-term  Soldier  in  their  MOS  who 
is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a  typical  unit 
assignment  and  position  and  at  least  one  series  of  Green-Amber-Red  training  cycles  and  unit 
operational  requirements.'’’’  Participants  were  then  provided  descriptions  of  the  scale(s)  on  which 
they  would  make  their  ratings. 

At  the  conclusion  of  each  questionnaire,  participants  were  asked  to  provide  feedback  on 
the  questionnaire’s  (a)  ease  of  use,  (b)  instructions,  (c)  clarity  of  the  statements  they  were  asked 
to  rate,  (d)  relevance  to  their  MOS,  and  (e)  usefulness  for  describing  the  requirements  of  their 
MOS.  For  each  of  these  five  dimensions,  participants  provided  ratings  on  a  1  (Very  bad)  to  5 
( Very  good)  scale.  Additionally,  participants  were  asked  to  respond  to  three  open-ended 
questions  regarding  their  ratings  and  their  reactions  to  the  questionnaire. 

Response  Rates  and  Completion  Times 

Response  rates  for  the  field  test  blocks  are  reported  in  Tables  7  and  8.  The  response  rates 
by  MOS  ranged  from  4.5%  to  20.2%,  with  many  rates  being  less  than  10%.  Among  the  MOS, 
25U  evidenced  the  highest  response  rates  across  all  four  questionnaire  blocks,  perhaps  owing  to 
participants’  greater  accessibility  to  an  Internet  connection  or  more  frequent  usage  of  their  AKO 
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accounts  compared  to  Soldiers  in  the  other  MOS.  The  percentage  of  participants  that  completed 
the  questionnaires  relative  to  those  accessing  it  was  considerably  higher  at  50%  or  greater. 
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Table7.  Response  Rates  for  Work  Activity  Questionnaire  Blocks  by  MOS  and  Rater  Type 


Accessed 

Completed 

Questionnaires 

Questionnaires 

N 

% 

% 

% 

MOS/Rater  Type 

Solicited 

N 

Solicited 

N 

Accessed 

Solicited 

Block  1  -  Work  Activities,  Abilities,  and  Work  Styles  Questionnaires 

11B 

706 

62 

8.78% 

32 

51.6% 

4.5% 

Incumbents 

335 

7 

2.09% 

4 

57.1% 

1.2% 

Supervisors 

371 

55 

14.82% 

28 

50.9% 

7.5% 

19K 

767 

35 

4.56% 

34 

97.1% 

4.4% 

Incumbents 

367 

8 

2.18% 

5 

62.5% 

1.4% 

Supervisors 

400 

27 

6.75% 

19 

70.4% 

4.8% 

25U 

746 

125 

16.76% 

78 

62.4% 

10.5% 

Incumbents 

373 

32 

8.58% 

21 

65.6% 

5.6% 

Supervisors 

373 

93 

24.93% 

57 

61.3% 

15.3% 

31B 

745 

94 

12.62% 

62 

66.0% 

8.3% 

Incumbents 

372 

19 

5.11% 

12 

63.2% 

3.2% 

Supervisors 

373 

73 

19.57% 

49 

67.1% 

13.1% 

63B 

775 

56 

7.23% 

36 

64.3% 

4.6% 

Incumbents 

396 

12 

3.03% 

6 

50.0% 

1.5% 

Supervisors 

379 

43 

11.35% 

30 

69.8% 

7.9% 

88M 

709 

70 

9.87% 

49 

70.0% 

6.9% 

Incumbents 

358 

14 

3.91% 

9 

64.3% 

2.5% 

Supervisors 

370 

55 

14.86% 

39 

70.9% 

10.5% 

Totals 

4,448 

442 

9.94% 

291 

65.84% 

6.54% 

Incumbents 

2,201 

92 

4.18% 

57 

61.96% 

2.59% 

Supervisors 

2,266 

346 

15.27% 

222 

64.16% 

9.80% 

Block  2  -  Work  Activities,  Work  Interests,  and  Work  Values  Questionnaires 

11B 

702 

65 

9.26% 

35 

53.8% 

5.0% 

Incumbents 

336 

11 

3.27% 

5 

45.5% 

1.5% 

Supervisors 

366 

54 

14.75% 

30 

55.6% 

8.2% 

19K 

750 

34 

4.53% 

25 

73.5% 

3.3% 

Incumbents 

350 

11 

3.14% 

9 

81.8% 

2.6% 

Supervisors 

400 

23 

5.75% 

16 

69.6% 

4.0% 

25U 

746 

128 

17.16% 

85 

66.4% 

11.4% 

Incumbents 

373 

35 

9.38% 

20 

57.1% 

5.4% 

Supervisors 

373 

93 

24.93% 

65 

69.9% 

17.4% 

31B 

745 

83 

11.14% 

48 

57.8% 

6.4% 

Incumbents 

372 

9 

2.42% 

6 

66.7% 

1.6% 

Supervisors 

373 

72 

19.30% 

42 

58.3% 

11.3% 

63B 

781 

43 

5.51% 

29 

67.4% 

3.7% 

Incumbents 

396 

13 

3.28% 

8 

61.5% 

2.0% 

Supervisors 

385 

30 

7.79% 

21 

70.0% 

5.5% 

88M 

728 

66 

9.07% 

48 

72.7% 

6.6% 

Incumbents 

345 

13 

3.77% 

10 

76.9% 

2.9% 

Supervisors 

364 

53 

14.56% 

38 

71.7% 

10.4% 

Totals 

4,452 

419 

9.41% 

270 

64.44% 

6.06% 

Incumbents 

2,172 

92 

4.24% 

58 

63.04% 

2.67% 

Supervisors 

2,261 

325 

14.37% 

212 

65.23% 

9.38% 

21 


Table  8.  Response  Rates  for  Work  Context  Questionnaire  Blocks  by  MOS  and  Rater  Type 


Accessed  Completed 

Questionnaires  _ Questionnaires 


N 

% 

% 

% 

MOS/Rater  Type 

Solicited 

N 

Solicited 

N 

Accessed 

Solicited 

Block  3  -  Work  Context,  Abilities,  and  Work  Styles  Questionnaires 

11B 

706 

66 

9.3% 

47 

71.2% 

6.7% 

Incumbents 

339 

17 

5.0% 

10 

58.8% 

2.9% 

Supervisors 

367 

48 

13.1% 

36 

77.1% 

9.8% 

19K 

767 

72 

9.4% 

50 

69.4% 

6.5% 

Incumbents 

367 

6 

1.6% 

3 

50.0% 

0.8% 

Supervisors 

400 

64 

16.0% 

46 

71.9% 

11.5% 

25U 

746 

201 

26.9% 

129 

64.2% 

17.3% 

Incumbents 

373 

52 

13.9% 

34 

65.4% 

9.1% 

Supervisors 

373 

145 

38.9% 

93 

64.1% 

24.9% 

31B 

744 

82 

11.0% 

54 

65.9% 

7.3% 

Incumbents 

372 

12 

3.2% 

9 

75.0% 

2.4% 

Supervisors 

372 

69 

18.5% 

45 

65.2% 

12.1% 

63B 

775 

78 

10.1% 

50 

64.1% 

6.5% 

Incumbents 

396 

11 

2.8% 

6 

54.5% 

1.5% 

Supervisors 

379 

67 

17.7% 

44 

65.7% 

11.6% 

88M 

705 

131 

18.6% 

80 

61.1% 

11.3% 

Incumbents 

339 

38 

11.2% 

19 

50.0% 

5.6% 

Supervisors 

366 

91 

24.9% 

61 

67.0% 

16.7% 

Totals 

4,443 

630 

14.18% 

410 

65.08% 

9.23% 

Incumbents 

2,186 

136 

6.22% 

81 

59.56% 

3.71% 

Supervisors 

2,257 

484 

21.44% 

325 

67.15% 

14.40% 

Block  4  -  Work  Context,  Work  Interests,  and  Work  Values  Questionnaires 

11B 

707 

54 

7.6% 

38 

70.4% 

5.4% 

Incumbents 

335 

12 

3.6% 

5 

41.7% 

1.5% 

Supervisors 

372 

42 

11.3% 

33 

78.6% 

8.9% 

19K 

750 

59 

7.9% 

42 

71.2% 

5.6% 

Incumbents 

350 

6 

1.7% 

5 

83.3% 

1.4% 

Supervisors 

400 

53 

13.3% 

37 

69.8% 

9.3% 

25U 

746 

223 

29.9% 

151 

67.7% 

20.2% 

Incumbents 

373 

56 

15.0% 

37 

66.1% 

9.9% 

Supervisors 

373 

167 

44.8% 

114 

68.3% 

30.6% 

31B 

745 

88 

11.8% 

63 

71.6% 

8.5% 

Incumbents 

372 

19 

5.1% 

12 

63.2% 

3.2% 

Supervisors 

373 

67 

18.0% 

51 

76.1% 

13.7% 

63B 

781 

61 

7.8% 

39 

63.9% 

5.0% 

Incumbents 

396 

18 

4.5% 

9 

50.0% 

2.3% 

Supervisors 

385 

43 

11.2% 

30 

69.8% 

7.8% 

88M 

706 

143 

20.2% 

96 

67.1% 

13.6% 

Incumbents 

338 

24 

7.1% 

11 

45.8% 

3.3% 

Supervisors 

368 

119 

32.3% 

85 

71.4% 

23.1% 

Totals 

4,435 

628 

14.16% 

429 

68.31% 

9.67% 

Incumbents 

2,164 

135 

6.24% 

79 

58.52% 

3.65% 

Supervisors 

2,271 

491 

21.62% 

350 

71.28% 

15.41% 
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Tables  9  and  10  reflect  the  completion  rates  by  questionnaire,  broken  out  by  MOS  and 
rater  type.  Both  raw  numbers  and  percentages  are  reported  to  determine  whether  there  were  large 
differences  in  the  completion  rates  between  MOS  and  within  MOS  by  rater  type.  Because 
portions  of  the  analyses  are  dependent  on  sample  size  (e.g.,  indices  of  reliability),  it  was 
important  to  determine  the  comparative  sample  sizes  for  each  questionnaire  included  in  the 
research.  Completion  rates  reflected  a  less  than  ideal  split  between  the  incumbents  and 
supervisors.  Even  though  the  solicited  sample  provided  by  HRC  oversampled  incumbents  to  the 
extent  feasible  (a  60-40%  was  desired),  the  splits  in  the  obtained  sample  were  generally  around 
20%-80%  between  incumbents  and  supervisors. 

Table  9.  Completed  Work-Oriented  Questionnaires  (Work  Activities  and  Work  Context) 


MOS/Rater  Type 

Work  Activities 

Work  Context 

N 

% 

N 

% 

1  IB 

67 

85 

Incumbents 

9 

13.4% 

15 

17.6% 

Supervisors 

58 

86.6% 

70 

82.4% 

19K 

49 

92 

Incumbents 

14 

28.6% 

8 

8.8% 

Supervisors 

35 

71.4% 

83 

91.2% 

25U 

163 

280 

Incumbents 

41 

25.2% 

71 

25.4% 

Supervisors 

122 

74.8% 

207 

73.9% 

31B 

no 

117 

Incumbents 

18 

16.4% 

21 

18.0% 

Supervisors 

91 

82.7% 

96 

82.0% 

63B 

65 

89 

Incumbents 

14 

21.5% 

15 

16.8% 

Supervisors 

51 

78.5% 

74 

83.2% 

88M 

97 

176 

Incumbents 

19 

19.6% 

30 

17.0% 

Supervisors 

77 

79.4% 

146 

83.0% 

Totals 

551 

839 

Incumbents 

115 

20.9% 

160 

19.1% 

Supervisors 

434 

78.8% 

676 

80.6% 
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Table  10.  Completed  Worker-Oriented  Questionnaires  (Abilities,  Work  Styles,  Work  Interests, 
Work  Values) 


Work  Abilities 

Work  Styles 

Work  Values 

Work  Interests 

MOS/Rater  Type 

N 

% 

N 

% 

N 

% 

N 

% 

1  IB 

79 

79 

73 

73 

Incumbents 

14 

17.7% 

14 

17.7% 

10 

13.7% 

10 

13.7% 

Supervisors 

65 

82.3% 

65 

82.3% 

63 

86.3% 

63 

86.3% 

19K 

74 

74 

67 

67 

Incumbents 

8 

11.0% 

8 

11.0% 

14 

20.9% 

14 

20.9% 

Supervisors 

65 

89.0% 

65 

89.0% 

53 

79.1% 

53 

79.1% 

25U 

207 

207 

236 

236 

Incumbents 

55 

26.8% 

55 

26.8% 

57 

24.2% 

57 

24.2% 

Supervisors 

150 

73.2% 

150 

73.2% 

179 

75.9% 

179 

75.9% 

31B 

116 

116 

111 

111 

Incumbents 

21 

18.3% 

21 

18.3% 

18 

16.2% 

18 

16.2% 

Supervisors 

94 

81.7% 

94 

81.7% 

93 

83.8% 

93 

83.8% 

63B 

86 

86 

68 

68 

Incumbents 

12 

14.0% 

12 

14.0% 

17 

25.0% 

17 

25.0% 

Supervisors 

74 

86.1% 

74 

86.1% 

51 

75.0% 

51 

75.0% 

88M 

129 

129 

144 

144 

Incumbents 

28 

21.9% 

28 

21.9% 

21 

14.6% 

21 

14.6% 

Supervisors 

100 

78.1% 

100 

78.1% 

123 

85.4% 

123 

85.4% 

Totals 

691 

691 

699 

699 

Incumbents 

138 

20.0% 

138 

20.0% 

137 

19.6% 

137 

19.6% 

Supervisors 

548 

79.3% 

548 

79.3% 

562 

80.4% 

562 

80.4% 

Completion  times  by  questionnaire  block,  MOS,  and  rater  type  are  summarized  in  Table 
1 1 .  Consistent  with  our  target,  the  average  time  to  complete  each  field  test  block  was  generally 
less  than  20  minutes.  Participants  took  the  longest  to  complete  Blocks  1  and  2,  which  included 
the  Work  Activities  questionnaire,  generally  the  longest  measures  of  those  administered.  Each 
consisted  of  more  than  30  items  and  required  three  ratings  per  item.  Across  all  blocks, 
incumbents  and  supervisors  within  an  MOS  took  similar  amounts  of  time  to  complete  the 
surveys.  The  largest  MOS  differences  in  completion  times  were  for  Blocks  1  and  2  (e.g.,  1  IB 
took  longer  than  all  other  MOS),  which  is  likely  attributable  to  different  lengths  in  the  Work 
Activities  questionnaires. 
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Table  11.  Completion  Times  by  Questionnaire  Block,  MOS,  and  Rater  Type 


Completion  Time  (in  Minutes) 

N 

90th 

MOS/Rater  Type 

Completed 

M 

Med 

SD  Percentile 

Block  1 

-  Work  Activities,  Abilities,  and  Work  Styles  Questionnaires 

1  IB 

32 

18.23 

18.58 

3.00 

21.90 

Incumbents 

4 

19.64 

21.07 

3.48 

21.93 

Supervisors 

28 

18.03 

18.47 

2.94 

21.85 

19K 

24 

11.09 

11.06 

2.07 

13.63 

Incumbents 

5 

10.99 

10.85 

2.14 

12.97 

Supervisors 

19 

11.11 

11.18 

2.10 

14.30 

25U 

78 

13.32 

13.84 

3.13 

17.07 

Incumbents 

21 

12.76 

13.60 

3.50 

16.92 

Supervisors 

57 

13.53 

13.88 

2.99 

17.14 

31B 

62 

14.86 

15.03 

2.90 

18.05 

Incumbents 

12 

15.14 

14.96 

3.52 

21.35 

Supervisors 

49 

14.78 

14.95 

2.80 

18.00 

63B 

36 

11.20 

11.59 

3.12 

14.97 

Incumbents 

6 

8.81 

7.38 

3.48 

13.62 

Supervisors 

30 

11.68 

11.63 

2.87 

15.54 

88M 

49 

11.67 

12.15 

2.85 

15.35 

Incumbents 

9 

12.89 

13.37 

2.58 

16.60 

Supervisors 

39 

11.47 

11.80 

2.86 

15.08 

Block  2  -  Work  Activities,  Work  Interests,  and  Work  Values  Questionnaires 

1  IB 

35 

17.95 

18.73 

3.64 

22.33 

Incumbents 

5 

16.44 

15.77 

3.07 

21.23 

Supervisors 

30 

18.21 

19.30 

3.71 

22.72 

19K 

25 

12.09 

12.25 

3.09 

16.09 

Incumbents 

9 

11.38 

11.73 

3.00 

15.30 

Supervisors 

16 

12.48 

12.48 

3.16 

16.63 

25U 

85 

14.49 

14.98 

3.35 

19.10 

Incumbents 

20 

14.52 

14.58 

3.56 

19.77 

Supervisors 

65 

14.48 

15.05 

3.31 

18.40 

31B 

48 

15.22 

15.98 

3.88 

20.21 

Incumbents 

6 

15.74 

16.28 

3.22 

19.15 

Supervisors 

42 

15.14 

15.98 

3.99 

20.65 

63B 

29 

15.22 

15.98 

3.88 

20.21 

Incumbents 

8 

10.66 

12.57 

3.85 

14.38 

Supervisors 

21 

11.09 

11.67 

3.13 

15.72 

88M 

48 

13.53 

13.80 

2.70 

17.24 

Incumbents 

10 

12.58 

13.38 

2.45 

15.37 

Supervisors 

38 

13.78 

14.06 

2.73 

17.34 

25 


Table  11.  (continued) 


Completion  Time  (in  Minutes) 

N 

90th 

MOS/Rater  Type 

Completed 

M 

Med 

SD  Percentile 

Block  3  -  Work  Context,  Abilities,  and  Work  Styles  Questionnaires 

1  IB 

47 

11.59 

11.73 

2.97 

15.03 

Incumbents 

10 

11.38 

11.83 

4.09 

17.51 

Supervisors 

36 

11.66 

11.80 

2.70 

14.51 

19K 

50 

11.11 

11.90 

2.73 

14.55 

Incumbents 

3 

9.22 

9.92 

3.37 

12.18 

Supervisors 

46 

11.19 

11.90 

2.70 

14.63 

25U 

129 

11.00 

11.45 

2.89 

14.15 

Incumbents 

34 

10.49 

10.37 

3.04 

14.18 

Supervisors 

93 

11.22 

11.47 

2.85 

13.93 

31B 

54 

11.76 

11.74 

2.28 

15.03 

Incumbents 

9 

11.79 

11.55 

2.07 

15.82 

Supervisors 

45 

11.76 

11.77 

2.34 

15.00 

63B 

50 

11.18 

11.37 

3.08 

14.85 

Incumbents 

6 

10.22 

10.54 

3.33 

13.50 

Supervisors 

44 

11.32 

11.64 

3.06 

15.18 

88M 

80 

11.52 

11.98 

2.56 

14.65 

Incumbents 

19 

11.84 

12.30 

2.02 

14.65 

Supervisors 

61 

11.42 

11.83 

2.71 

14.83 

Block  4  - 

Work  Context,  Work  Interests,  and  Work  Values  Questionnaires 

1  IB 

38 

12.51 

13.16 

3.14 

16.22 

Incumbents 

5 

12.72 

12.43 

3.43 

16.23 

Supervisors 

33 

12.48 

13.42 

3.15 

16.21 

19K 

42 

11.77 

12.25 

2.85 

15.16 

Incumbents 

5 

10.15 

8.87 

3.60 

15.65 

Supervisors 

37 

11.99 

12.62 

2.72 

15.00 

25U 

151 

12.61 

12.90 

3.02 

15.85 

Incumbents 

37 

12.80 

13.12 

3.30 

17.06 

Supervisors 

114 

12.55 

12.90 

2.94 

15.63 

31B 

63 

12.09 

12.52 

2.78 

15.63 

Incumbents 

12 

10.49 

10.58 

2.59 

14.29 

Supervisors 

51 

12.46 

12.65 

2.71 

15.82 

63B 

39 

12.61 

13.17 

2.56 

15.95 

Incumbents 

9 

12.24 

12.17 

1.77 

14.62 

Supervisors 

30 

12.72 

13.29 

2.77 

15.97 

88M 

96 

12.43 

12.93 

2.56 

15.17 

Incumbents 

11 

13.67 

13.87 

0.97 

14.67 

Supervisors 

85 

12.27 

12.83 

2.66 

15.22 
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Analysis  Sample  Demographics 


The  demographic  characteristics  of  the  field  test  sample  are  summarized  in  Table  12.  As 
noted  previously,  most  of  the  participants  were  supervisors  (N  =  1,111,  compared  to  N  =  277  for 
incumbents).  Additionally,  the  highest  percentage  of  participants  was  from  25U,  a  level  much 
higher  than  their  representation  in  the  Army  population  as  a  whole.  Because  of  these  sample  size 
differences,  the  analyses  were  conducted  by  rater  type  and  MOS  in  addition  to  reporting  the 
results  for  the  sample  as  a  whole. 


Table  12.  Demographic  Characteristics  of  Field  Test  Sample 


Incumbents 
(/V=  277) 

Supervisors 
(N=  1,111) 

Total 

(N=  1,390) 

Demographic 

N 

% 

N 

% 

N 

% 

MOS 

1  IB 

24 

8.7% 

128 

11.5% 

152 

10.9% 

19K 

22 

8.0% 

118 

10.6% 

141 

10.1% 

25U 

112 

40.7% 

329 

29.6% 

443 

31.8% 

63B 

39 

14.2% 

187 

16.8% 

227 

16.3% 

31B 

29 

10.5% 

125 

11.6% 

154 

11.1% 

88M 

49 

17.8% 

223 

20.1% 

273 

19.6% 

Rank 

PV2 

24 

8.66% 

N/A 

N/A 

24 

1.7% 

PFC 

93 

33.57% 

N/A 

N/A 

93 

6.7% 

SPC/CPL 

160 

57.76% 

N/A 

N/A 

160 

11.5% 

SGT 

N/A 

N/A 

351 

31.6% 

351 

25.2% 

SSG 

N/A 

N/A 

436 

39.2% 

436 

31.3% 

SFC 

N/A 

N/A 

316 

28.4% 

316 

22.7% 

Missing 

N/A 

N/A 

8 

0.7% 

14 

1.0% 

Months  in  Active  Army 

Between  12  and  18  months 

65 

23.5% 

4 

.4% 

69 

5.0% 

Between  18  and  24  months 

71 

25.6% 

6 

.5% 

77 

5.5% 

Between  24  and  36  months 

55 

19.9% 

19 

1.7% 

74 

5.3% 

Between  36  and  48  months 

32 

11.6% 

52 

4.7% 

84 

6.0% 

More  than  48  months 

54 

19.5% 

1,026 

92.4% 

1,085 

77.8% 

Missing 

0 

0 

4 

.4% 

5 

.4% 

Months  Assigned  to  Duty  Position 

6  months  or  less 

119 

42.9% 

216 

19.4% 

277 

19.9% 

Between  6  and  12  months 

99 

35.7% 

259 

23.3% 

334 

24.0% 

Between  12  and  18  months 

40 

14.3% 

205 

18.5% 

269 

19.3% 

Between  18  and  24  months 

19 

7.1% 

128 

11.5% 

167 

12.0% 

More  than  24  months 

0 

0 

302 

27.2% 

345 

24.8% 

Missing 

0 

0 

1 

.1% 

2 

.1% 

Organizational  Level 

Company/T  roop/B  attery 

181 

65.3% 

563 

50.7% 

745 

53.4% 

Battalion/Squadron 

52 

18.8% 

304 

27.4% 

357 

25.6% 

Regiment/Group 

2 

0.7% 

13 

1.2% 

15 

1.1% 

Brigade 

19 

6.9% 

94 

8.5% 

113 

8.1% 

Division 

9 

3.3% 

27 

2.4% 

39 

2.8% 

Corps 

1 

.4% 

5 

.5% 

6 

.4% 

Garrison 

3 

1.1% 

21 

1.9% 

24 

1.7% 

Installation 

2 

.7% 

15 

1.4% 

17 

1.2% 

FIQ  Army  Command 

2 

.7% 

9 

.8% 

11 

.8% 

F1Q  Army  Service  Component 

0 

0 

2 

.2% 

2 

.1% 
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Table  12.  (continued) 


Incumbents 
(A  =277) 

Supervisors 

0V=U11) 

(N 

Total 
=  1,390) 

Demographic 

N 

% 

N 

% 

N 

% 

HQ  Direct  Reporting  Unit 

2 

.7% 

0 

0 

2 

.1% 

HQ  Department  of  the  Army 

0 

0 

5 

.5% 

5 

.4% 

None  of  the  levels  listed 

4 

1.4% 

49 

4.4% 

54 

3.8% 

Missing 

0 

0 

1 

.1% 

1 

.1% 

Months  Deployed 

None 

113 

40.8% 

72 

6.5% 

186 

13.3% 

6  months  or  less 

19 

6.9% 

26 

2.3% 

45 

3.2% 

Between  6  and  15  months 

98 

35.4% 

213 

19.2% 

311 

22.3% 

Between  15  and  24  months 

30 

10.8% 

268 

24.1% 

301 

21.6% 

Between  24  and  36  months 

17 

6.1% 

381 

34.3% 

399 

28.6% 

More  than  36  months 

0 

0 

151 

13.6% 

152 

10.9% 

Missing 

0 

0 

0 

0 

0 

0 

Results 


Basic  Descriptive  Statistics 

Two  statistics  describing  incumbent  and  supervisor  ratings  of  the  survey  items  are  reported 
in  Tables  13  through  17:  (a)  the  mean  of  the  item  means,  and,  (b)  the  median  of  the  item  standard 
deviations  ( SDs ).  The  mean  of  the  item  means  (mean  of  Ms)  reflects  the  average  value  of  the 
responses  to  items  comprising  each  of  the  respective  questionnaire  domains.  It  is  calculated  by 
taking  the  average  of  each  item  response,  segmented  by  rater  type  and  MOS,  and  weighting  the 
responses  by  the  number  of  respondents  who  completed  each  item.  The  median  of  the  item  SDs 
represents  the  50th  percentile.  It  is  computed  by  taking  the  median  (midpoint)  of  the  range  of  SDs 
across  items.  It  reflects  the  variation  in  responses  to  each  of  the  domain  items,  both  between  MOS 
and  within  MOS  by  rater  type.  The  median  was  selected  because  the  SDs  within  MOS  and  rater 
type  could  be  easily  influenced  by  small  sample  sizes;  reporting  the  mean  SD  would  overweight 
SDs  that  were  aberrantly  large  and  reflect  an  inflated  degree  of  variation  in  responses.  In  addition 
to  reporting  the  results  by  MOS  and  rater  type,  we  also  report  the  total  sample-weighted  means  for 
the  entire  sample  and  by  rater  type.  We  elected  to  report  the  mean  of  the  median  SDs  for  the  entire 
sample,  as  the  median  SDs  contributing  to  the  mean  value  already  accounted  for  any  aberrant 
responses  or  small  sample  sizes. 

Work-Oriented  Descriptor  Ratings 

The  results  of  the  analysis  of  the  work-oriented  descriptors  (e.g.,  work  activities,  work 
context)  can  be  found  in  Tables  13,  14,  and  15.  Work  activity  descriptors  were  rated  on  three 
scales:  (a)  importance,  (b)  training  difficulty,  and  (c)  criticality.  Participants  completed  work 
activity  items  specific  to  their  MOS.  The  means  for  importance  ratings  ranged  from  3.86  to  4.50, 
with  the  sample-weighted  mean  rating  of  4.09  ( SD  =  1 .09).  The  means  for  training  difficulty 
ratings  ranged  from  2.27  to  3.08,  while  the  sample-weighted  mean  was  2.71  (SD  =  1.17). 

Finally,  the  means  of  criticality  ratings  ranged  from  3.76  to  4.42,  while  the  sample-weighted 
mean  was  4.01  (SD  =  1.14). 
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Table  13.  Descriptive  Statistics  for  Ratings  of  Work  Activities  by  MOS  and  Rater  Type 


MOS/Rater  Type 

N 

Number  of 
Statements 

Importance  (1-5) 

N 

Trainability  (1-5) 

N 

Criticality  (1-5) 

Mean  of 
Ms 

Mdn  of 
SDs 

Mean  of 
Ms 

Mdn  of 
SDs 

Mean  of 
Ms 

Mdn  of 
SDs 

11B 

64-66 

64 

4.44 

.79 

64-66 

2.72 

1.14 

64-66 

4.37 

.88 

Incumbents 

9 

64 

4.47 

.73 

9 

2.41 

1.01 

8-9 

4.42 

1.00 

Supervisors 

55-57 

64 

4.44 

.81 

55-57 

2.76 

1.14 

55-57 

4.36 

.91 

19K 

45-48 

31 

4.17 

.99 

46-48 

2.74 

1.12 

45-48 

4.08 

1.08 

Incumbents 

13-14 

31 

4.50 

.77 

13-14 

2.55 

1.27 

13-14 

4.32 

1.02 

Supervisors 

31-34 

31 

4.03 

1.00 

32-34 

2.79 

1.06 

31-34 

3.97 

1.12 

25U 

159-161 

31 

3.96 

1.16 

159-161 

2.80 

1.13 

159-161 

3.91 

1.23 

Incumbents 

39-40 

31 

3.84 

1.28 

40 

2.72 

1.14 

40 

3.76 

1.36 

Supervisors 

119-121 

31 

4.00 

1.13 

119-121 

2.83 

1.13 

119-121 

3.96 

1.18 

31B 

104-108 

43 

3.90 

1.16 

104-108 

2.81 

1.12 

104-108 

3.81 

1.19 

Incumbents 

17-18 

43 

4.11 

1.24 

18 

2.62 

1.18 

17-18 

3.98 

1.32 

Supervisors 

85-89 

43 

3.86 

1.14 

85-89 

2.84 

1.10 

85-89 

3.78 

1.19 

63B 

62-63 

18 

4.11 

1.16 

62-63 

2.41 

1.27 

61-63 

4.08 

1.11 

Incumbents 

14 

18 

4.33 

.91 

14 

2.35 

1.23 

14 

4.13 

.92 

Supervisors 

48-49 

18 

4.05 

1.25 

48-49 

2.43 

1.29 

47-49 

4.06 

1.17 

88M 

91-95 

25 

4.26 

1.13 

91-95 

2.91 

1.27 

91-95 

4.09 

1.19 

Incumbents 

19 

25 

4.11 

1.22 

19 

2.29 

1.23 

19 

3.86 

1.39 

Supervisors 

71-75 

25 

4.31 

1.09 

71-75 

3.08 

1.23 

71-75 

4.16 

1.13 

Total  M 

4.09 

1.09 

2.71 

1.17 

4.01 

1.14 

Incumbent  M 

4.08 

1.10 

2.54 

1.18 

3.97 

1.24 

Supervisor  M 

4.09 

1.09 

2.82 

1.16 

4.02 

1.14 

Note.  Ns  are  reported  as  ranges  because  Mean  and  SD  information  is  derived  from  individual  items.  Total  M,  Incumbent  M,  and  Supervisor  M  are  sample- 
weighted  means. 


Table  14.  Descriptive  Statistics  for  Work  Activities  Category  Ratings  by  MOS  and  Rater  Type 


MOS/Rater  Type 

N 

Number  of 
Statements 

Importance  (1-5) 

N 

Trainability  (1-5) 

N 

Criticality  (1-5) 

Mean  of 
Ms 

Mdn  of 
SDs 

Mean  of 
Ms 

Mdn  of 
SDs 

Mean  of 
Ms 

Mdn  of 
SDs 

1  IB 

64-66 

13 

4.46 

.68 

64-66 

2.84 

1.17 

63-66 

4.42 

.75 

Incumbents 

8-9 

13 

4.48 

.73 

8-9 

2.34 

1.00 

8-9 

4.48 

.73 

Supervisors 

55-57 

13 

4.46 

.68 

55-57 

2.92 

1.17 

54-57 

4.41 

.76 

19K 

47-48 

10 

4.28 

.94 

47-48 

2.72 

1.06 

47-48 

4.24 

.96 

Incumbents 

13-14 

10 

4.50 

1.02 

14 

2.36 

1.21 

14 

4.41 

1.06 

Supervisors 

34 

10 

4.19 

.98 

33-34 

2.87 

.97 

33-34 

4.17 

.90 

25U 

154-161 

7 

4.04 

1.05 

155-161 

2.91 

1.09 

155-161 

4.09 

1.06 

Incumbents 

38-40 

7 

3.92 

1.22 

39-40 

2.94 

1.22 

39-40 

3.98 

1.25 

Supervisors 

116-121 

7 

4.07 

.97 

116-121 

2.91 

1.08 

116-121 

4.13 

.97 

31B 

105-108 

8 

4.00 

1.10 

105-108 

2.96 

1.06 

105-108 

4.01 

1.12 

Incumbents 

18 

8 

4.26 

1.10 

18 

2.97 

1.28 

17-18 

4.19 

1.16 

Supervisors 

86-89 

8 

3.95 

1.09 

86-89 

2.95 

1.04 

86-89 

3.97 

1.11 

63B 

62-63 

6 

4.21 

1.03 

62-63 

2.53 

1.35 

62-63 

4.23 

1.06 

Incumbents 

14 

6 

4.45 

.80 

14 

2.28 

1.34 

13-14 

4.40 

1.04 

Supervisors 

48-49 

6 

4.14 

1.11 

48-49 

2.60 

1.32 

48-49 

4.18 

1.07 

88M 

91-95 

6 

4.29 

1.10 

91-95 

2.88 

1.25 

91-95 

4.23 

1.11 

Incumbents 

17-19 

6 

4.16 

1.28 

17-19 

2.04 

1.11 

17-19 

3.92 

1.41 

Supervisors 

73-75 

6 

4.33 

1.07 

73-75 

3.09 

1.21 

73-75 

4.32 

1.04 

Total  M 

4.17 

1.01 

2.84 

1.15 

4.17 

1.03 

Incumbent  M 

4.20 

1.09 

2.60 

1.21 

4.15 

1.17 

Supervisor  M 

4.16 

.99 

2.91 

1.12 

4.18 

.99 

Note.  Ns  are  reported  as  ranges  because  Mean  and  SD  information  is  derived  from  individual  items.  Total  M,  Incumbent  M,  and  Supervisor  M  are  sample- 
weighted  means. 


As  with  the  individual  work  activity  descriptors,  the  work  activity  categories  were  also 
MOS-specific  and  rated  on  the  same  three  scales.  As  shown  in  Table  14,  the  means  of 
importance  ratings  ranged  from  3.92  to  4.50,  with  the  sample-weighted  mean  being  4.17  ( SD  = 
1.01).  The  means  of  training  difficulty  ratings  ranged  from  2.28  to  3.09,  while  the  sample- 
weighted  mean  was  2.84  ( SD  =  1.15).  Finally,  the  means  of  the  criticality  ratings  ranged  from 
3.92  to  4.48,  while  the  sample-weighted  mean  was  4.17  (SD  =  1.03). 

We  would  not  expect  the  training  and  either  the  importance  or  criticality  ratings  to  be 
systematically  correlated  and  they  were  not.  The  lowest  correlation  among  ratings  of  the 
individual  (as  opposed  to  category)  work  activities  was  between  incumbent  criticality  and 
training  (r  =  -.03)  and  the  highest  correlation  was  between  supervisor  criticality  and  training 
ratings  (r  =  .20).  We  did  expect  that  the  importance  and  criticality  ratings  would  be  highly 
correlated  and  indeed  they  were  (r  =  .92  for  incumbents  and  r  =  .97  for  supervisors). 


Table  15.  Descriptive  Statistics  for  Work  Context  Ratings 


MOS/Rater 

Type 

N 

Number 

ofWCs 

Non-Combat 
Ratings  (1-5) 

Combat  Ratings 
(1-5) 

Mean 
of  Ms 

Mdn  of 
SDs 

N 

Mean  of 
Ms 

Mdn  of 
SDs 

1  IB 

74-76 

46 

2.63 

1.35 

74-76 

3.64 

1.19 

Incumbents 

13 

46 

2.93 

1.54 

12-13 

3.88 

1.08 

Supervisors 

61-63 

46 

2.57 

1.31 

60-62 

3.59 

1.19 

19K 

85-87 

46 

2.68 

1.23 

86-87 

3.62 

1.18 

Incumbents 

7 

46 

3.06 

1.70 

6-7 

3.29 

1.73 

Supervisors 

77-79 

46 

2.64 

1.17 

78-79 

3.64 

1.14 

25U 

254-257 

46 

2.51 

1.29 

254-257 

3.39 

1.27 

Incumbents 

56 

46 

2.52 

1.39 

54-56 

3.25 

1.33 

Supervisors 

196-199 

46 

2.52 

1.24 

196-199 

3.43 

1.25 

31B 

106-107 

46 

2.89 

1.28 

106-107 

3.52 

1.25 

Incumbents 

18-19 

46 

2.67 

1.35 

18-19 

2.93 

1.51 

Supervisors 

87-88 

46 

2.94 

1.24 

87-88 

3.64 

1.15 

63B 

73-74 

46 

2.57 

1.32 

73-74 

3.30 

1.28 

Incumbents 

9-10 

46 

2.49 

1.45 

9-10 

3.24 

1.38 

Supervisors 

64 

46 

2.58 

1.27 

64 

3.31 

1.27 

88M 

148-150 

46 

2.37 

1.29 

147-150 

3.47 

1.29 

Incumbents 

19-20 

46 

2.62 

1.55 

18-20 

3.54 

1.44 

Supervisors 

128-130 

46 

2.33 

1.23 

128-130 

3.45 

1.25 

Total  M 

2.57 

1.30 

3.47 

1.28 

Incumbent  M 

2.63 

1.44 

3.32 

1.42 

Supervisor  M 

2.57 

1.27 

3.50 

1.29 

Note.  Ns  are  reported  as  ranges  because  Mean  and  SD  information  is  derived  from  individual  items.  Total  M, 
Incumbent  M,  and  Supervisor  M  are  sample-weighted  means. 

Work  context  descriptors  were  rated  on  the  same  scale  in  two  settings:  (a)  non-combat 
and  (b)  combat.  The  means  of  the  non-combat  importance  ratings  ranged  from  2.33  to  3.06,  with 
the  sample-weighted  mean  being  2.57  (SD  =  1.30).  The  means  of  the  combat  ratings  ranged  from 
2.93  to  3.88,  with  the  sample  weighted  mean  being  3.47  (SD  =  1.28).  The  large  difference  in 
item  ratings  between  combat  and  non-combat  contexts  is  notable  and  consistent  with  previous 
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research.  Results  of  the  Army  0*NET  report  (Russell  et  ah,  2008)  demonstrated  that  when 
SMEs  rated  0*NET  work  context  descriptors,  the  ratings  needed  to  be  made  separately  for 
deployment  and  in-garrison  activities.  These  results  support  the  notion  that  both  incumbents  and 
supervisors  view  the  work  context  within  their  MOS  differently  based  on  whether  the  job  duties 
are  conducted  in  combat  or  non-combat  environments. 

Worker-Oriented  Descriptor  Ratings 

The  results  of  the  analysis  of  the  worker-oriented  descriptors  (e.g.,  work  abilities,  styles, 
values,  and  interests)  can  be  found  in  Tables  16  and  17.  The  means  for  work  abilities  ratings 
ranged  from  3.25  to  4.17,  with  the  sample-weighted  mean  being  3.68  ( SD  =  1.14).  The  means  for 
the  work  styles  ratings  ranged  from  3.23  to  4.61,  while  the  sample-weighted  mean  was  3.90  (SD 
=  1.04).  The  means  for  the  work  values  ratings  ranged  from  2.75  to  3.78,  while  the  sample- 
weighted  mean  was  3.54  (SD  =  1.23).  The  means  for  the  work  interests  ratings  ranged  from  3.22 
to  3.74,  while  the  sample-weighted  mean  was  3.46  (SD  =  1.31). 

Table  16.  Descriptive  Statistics  for  Work  Ability  and  Work  Style  Importance  Ratings 


MOS/Rater  Type 

Work  Abilities  (1-5) 

Work  Styles  (1-5) 

N 

Mean 
of  Ms 

Mdn  of 

SDs 

N 

Mean  of 
Ms 

Mdn  of 
SDs 

1  IB 

62-63 

3.83 

1.19 

55 

4.07 

1.03 

Incumbents 

10 

4.17 

1.35 

8 

4.61 

.74 

Supervisors 

52-53 

3.94 

1.08 

47 

3.93 

1.03 

19K 

56 

3.70 

.99 

48-49 

3.90 

.95 

Incumbents 

6 

4.03 

1.20 

4 

4.34 

.58 

Supervisors 

49 

3.69 

.98 

43-44 

3.87 

.95 

25U 

161-163 

3.61 

1.15 

136 

3.90 

.99 

Incumbents 

40 

3.25 

1.26 

37 

3.63 

1.16 

Supervisors 

121-123 

3.47 

1.14 

99 

4.00 

.90 

31B 

101-103 

3.83 

1.02 

93 

3.95 

.97 

Incumbents 

20 

3.96 

1.17 

16 

4.18 

.89 

Supervisors 

80-82 

3.78 

1.04 

76 

3.90 

.95 

63B 

76 

3.66 

1.20 

66-67 

3.73 

1.22 

Incumbents 

9 

3.31 

1.64 

8 

3.23 

1.73 

Supervisors 

67 

3.66 

1.12 

58-59 

3.80 

1.11 

88M 

96-98 

3.57 

1.16 

83-85 

3.88 

1.04 

Incumbents 

19-20 

3.70 

1.21 

17 

3.94 

1.06 

Supervisors 

75-77 

3.54 

1.16 

65-67 

3.89 

1.01 

Total  M 

3.68 

1.14 

3.90 

1.04 

Incumbent  M 

3.69 

1.27 

3.87 

1.18 

Supervisor  M 

3.68 

1.11 

3.99 

1.00 

Note.  Ns  are  reported  as  ranges  because  Mean  and  SD  information  is  derived  from  individual  items. 
Total  M,  Incumbent  M,  and  Supervisor  M  are  sample-weighted  means. 
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Table  1 7.  Descriptive  Statistics  for  Work  Values  and  Work  Interests  Importance  Ratings 


MOS/Rater  Type 

Work  Values  (1-5) 

Work  Interests  (1-5) 

N 

Mean  of 
Ms 

Mdn  of 
SDs 

N 

Mean  of 
Ms 

Mdn  of 
SDs 

1  IB 

49-50 

3.61 

1.22 

56-57 

3.37 

1.38 

Incumbents 

7 

3.05 

1.50 

8 

3.22 

1.53 

Supervisors 

42-43 

3.70 

1.18 

48-49 

3.39 

1.38 

19K 

47 

3.66 

1.13 

52-53 

3.44 

1.36 

Incumbents 

7 

3.78 

1.38 

10 

3.74 

1.41 

Supervisors 

40 

3.64 

1.09 

42-43 

3.37 

1.33 

25U 

168-170 

3.52 

1.20 

183-185 

3.52 

1.22 

Incumbents 

42 

3.45 

1.27 

44-45 

3.58 

1.25 

Supervisors 

126-128 

3.54 

1.17 

138-140 

3.50 

1.24 

31B 

74-75 

3.54 

1.20 

93-94 

3.60 

1.24 

Incumbents 

8 

2.75 

1.59 

13 

3.49 

1.52 

Supervisors 

66-67 

3.63 

1.12 

80-81 

3.62 

1.17 

63B 

43-44 

3.52 

1.25 

52-53 

3.30 

1.29 

Incumbents 

10-11 

3.63 

1.13 

14 

3.38 

1.24 

Supervisors 

32-33 

3.48 

1.28 

38-39 

3.27 

1.33 

88M 

102-104 

3.50 

1.32 

117-119 

3.39 

1.43 

Incumbents 

17-18 

3.44 

1.49 

18 

3.51 

1.45 

Supervisors 

84-86 

3.51 

1.29 

99-101 

3.36 

1.43 

Total  M 

3.54 

1.23 

3.46 

1.31 

Incumbent  M 

3.40 

1.36 

3.52 

1.34 

Supervisor  M 

3.57 

1.20 

3.45 

1.30 

Note.  Ns  are  reported  as  ranges  because  Mean  and  SD  information  is  derived  from  individual  items.  Total  M, 
Incumbent  M,  and  Supervisor  M  are  sample-weighted  means. 


Were  the  Jobs  Rated  Reliably  Using  the  Field  Test  Questionnaires? 

Two  metrics  are  reported  in  Tables  18  through  22:  (a)  the  single  rater  reliability  [ICC 
(C,l)],  and,  (b)  the  multi-rater  reliability  [ICC  (C,k)].  The  single  rater  reliability  index  is  a 
measure  of  reliability  based  on  the  ANOVA  framework;  it  is  computed  by  assessing  the  amount 
of  variance  in  ratings  within  group  (e.g.,  within  MOS)  in  comparison  to  the  amount  of  variance 
in  ratings  between  groups  (e.g.,  between  MOS).  The  multi-rater  reliability  index,  ICC  (C ,k)  is 
comparable  to  coefficient  alpha  and  is  a  measure  of  consistency  of  ratings  across  multiple  raters. 
We  report  the  results  for  ICC  (C,l)  and  ICC  (C ,k)  by  MOS  and  rater  type. 

Generally  speaking,  the  single  rater  reliability  provides  an  estimate  of  how  consistent  any 
single  rater  was  with  the  other  raters  in  the  group,  on  average.  Because  the  single-rater  reliability  is 
sample-size  invariant,  it  is  generally  used  to  make  cross-instrument  or  cross-sample  comparisons.  In 
contrast,  the  multi-rater  reliability  estimate  speaks  to  how  reliable  or  consistent  the  mean  rating 
supplied  by  that  group  of  raters  was  (i.e.,  what  is  the  likelihood  that  a  similar  but  different  group  of 
raters  from  the  same  population  would  produce  the  same  mean  rating).  For  most  applications, 
including  those  for  which  the  Army  would  be  using  these  data  (e.g.,  clustering  MOS),  the  most 
relevant  of  the  two  is  the  multi-rater  ICC,  because  operational  decisions  are  generally  based  on  the 
aggregate  judgment  of  a  group  (or  sample)  of  raters  on  which  the  data  were  collected. 
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Work-Oriented  Descriptor  Reliability  Estimates 

The  reliability  estimates  for  the  work-oriented  descriptors  are  presented  in  Tables  18  to 
20.  For  work  activities  ratings,  we  estimated  single  and  multi-rater  reliability  estimates  for  all 
three  types  of  ratings:  (a)  importance,  (b)  training  difficulty,  and,  (c)  criticality.  For  work 
context,  we  estimated  multi-rater  reliability  estimates  for  both  categories  of  ratings:  (a)  non¬ 
combat,  and  (b)  combat.  In  evaluating  the  multi-rater  reliability  estimates,  we  generally  focused 
on  comparing  the  estimates  in  this  research  to  the  0*NET  target  level  -  a  median  ICC  (C ,k)  of 
.80  or  greater  (McCloy,  Waugh,  &  Medsker,  1998). 

Reliability  estimates  for  the  work  activities  categories  are  presented  in  Table  18.  Across 
all  multi-rater  reliability  estimates,  the  0*NET  rule  of  thumb  of  .80  was  exceeded  (values  ranged 
from  .85  to  .99).  In  general,  there  were  no  systematic  differences  in  reliability  estimates  between 
MOS  across  all  activities  ratings  (values  ranged  from  .93  to  .98).  Additionally,  there  were  no 
systematic  differences  in  multi-rater  reliability  estimates  between  rater  types  across  activities 
ratings.  While  estimates  ranged  from  .85  (19K  incumbent  importance  ratings)  to  .99  (1  IB 
incumbent  trainability  ratings),  the  majority  of  values  were  .95  and  above.  Compared  to  the 
results  found  in  the  Army  0*NET  report,  we  generally  found  a  higher  supervisor  ICC  (C ,k)  for 
the  importance  ratings  for  all  four  MOS  included  in  both  projects  (i.e.,  19K,  25U,  3  IB,  and 
88M).  This  is  most  likely  due  to  the  larger  number  of  raters  included  in  the  present  research 
compared  to  the  Army  0*NET  study,  which  included  only  eight  raters  for  each  domain. 


Table  18.  ICCs  of  Work  Activities  Ratings  by  MOS  and  Rater  Type 


MOS/Rater 

Type 

N 

Number  of 
Statements 

Importance 

Army  0*NET 

Trainability 

Criticality 

ICC 

(C,1) 

ICC 

(C,k) 

ICC 

(C,l) 

ICC 
(C  ,k) 

ICC 

(C,l) 

ICC 
(C  ,k) 

ICC 

(C,l) 

ICC 

(C.k) 

1  IB 

57-60 

64 

.32 

.97 

.45 

.98 

.37 

.97 

Incumbents 

8-9 

64 

.27 

.96 

.55 

.99 

.35 

.97 

Supervisors 

49-51 

64 

.34 

.97 

.43 

.98 

.38 

.97 

19K 

38-49 

31 

.38 

.95 

.43 

.96 

.48 

.97 

Incumbents 

13 

31 

.16 

.85 

.41 

.96 

.42 

.96 

Supervisors 

25-27 

31 

.42 

.96 

.58 

.89 

.46 

.96 

.50 

.97 

25U 

152-154 

31 

.36 

.95 

.47 

.96 

.39 

.95 

Incumbents 

39-40 

31 

.42 

.96 

.51 

.97 

.50 

.97 

Supervisors 

112-114 

31 

.33 

.94 

.41 

.86 

.45 

.96 

.34 

.94 

31B 

98-102 

43 

.39 

.97 

.36 

.96 

.39 

.97 

Incumbents 

16-18 

43 

.37 

.96 

.38 

.96 

.34 

.96 

Supervisors 

81-83 

43 

.40 

.97 

.57 

.92 

.35 

.96 

.41 

.97 

63B 

58-62 

18 

.50 

.95 

.53 

.95 

.42 

.93 

Incumbents 

14 

18 

.39 

.92 

.55 

.96 

.35 

.91 

Supervisors 

44-48 

18 

.51 

.95 

.53 

.95 

.44 

.93 

88M 

86-87 

25 

.42 

.95 

.56 

.97 

.53 

.97 

Incumbents 

19 

25 

.32 

.92 

.58 

.97 

.45 

.95 

Supervisors 

66-67 

25 

.45 

.95 

.42 

.87 

.55 

.97 

.56 

.97 

Note.  Ns  are  reported  as  ranges  because  Mean  and  SD  information  is  reported  as  average  of  individual  items.  MOS 
analyzed  in  the  Army  0*NET  study  were  19K,  25U,  3  IB,  and  88M. 
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Whereas  the  multi-rater  estimates  were  higher  in  the  present  research,  the  single-rater 
work  activities  reliability  estimates  were  generally  higher  in  the  Army  0*NET  study  (Russell  et 
ah,  2008).  This  is  likely  attributable  to  the  more  intense  training  the  supervisors  received. 
Whereas  raters  in  this  research  were  untrained  and  unproctored,  the  Army  0*NET  study  used 
proctored  rating  sessions.  The  presence  of  a  proctor  may  have  increased  rater  motivation  to 
complete  the  ratings  in  a  more  consistent  manner. 

The  work  activities  categories  reliability  estimates  are  presented  in  Table  19.  All  multi¬ 
rater  reliability  estimates  were  above  the  0*NET  rule  of  thumb  with  one  exception  (see  88M 
incumbent  ratings  of  importance).  Multi-rater  reliability  estimates  ranged  from  .71  to  .91.  In 
general,  there  were  no  systematic  differences  in  the  reliability  estimates  by  MOS  across  all 
activities  category  ratings;  these  estimates  ranged  from  .84  to  .91.  For  the  most  part,  there  were 
no  systematic  differences  in  multi-rater  reliability  estimates  between  rater  types  across  all 
category  ratings  (values  ranged  from  .71  to  .92).  However,  supervisors  generally  rated  the 
importance,  trainability,  and  criticality  items  somewhat  more  reliably  than  did  incumbents. 
Although  not  directly  comparable  to  the  importance  ratings  found  in  the  Anny  0*NET  report, 
we  found  almost  identical  supervisor  multi-rater  reliability  estimates  for  the  work  activity 
category  importance  ratings  for  the  four  MOS  examined  in  both  pieces  of  research. 


Table  1 9.  ICCs  of  Work  Activities  Category  Ratings  by  MOS  and  Rater  Type 


MOS/Rater 

Type 

N 

Number  of  - 
Statements 

Importance 

Army  0*NET 

Trainability 

Criticality 

ICC 

(C,l) 

ICC 

(C,k) 

ICC 

(C,l) 

ICC 
(C  ,k) 

ICC 

(C,l) 

ICC 
(C  ,k) 

ICC 

(C,l) 

ICC 
(C  ,k) 

1  IB 

61-63 

13 

.41 

.90 

.67 

.96 

.44 

.91 

Incumbents 

8 

13 

.31 

.85 

.70 

.97 

.35 

.88 

Supervisors 

53-55 

13 

.43 

.91 

.66 

.96 

.46 

.92 

19K 

47 

10 

.42 

.88 

.47 

.90 

.42 

.88 

Incumbents 

13-14 

10 

.31 

.82 

.42 

.88 

.38 

.86 

Supervisors 

33-34 

10 

.45 

.89 

.58 

.89 

.47 

.90 

.46 

.89 

25U 

152-154 

7 

.48 

.87 

.55 

.90 

.44 

.84 

Incumbents 

38-39 

7 

.51 

.88 

.53 

.89 

.50 

.88 

Supervisors 

114-115 

7 

.46 

.86 

.41 

.86 

.57 

.90 

.41 

.83 

31B 

101-103 

8 

.57 

.91 

.58 

.92 

.55 

.91 

Incumbents 

17-18 

8 

.48 

.88 

.63 

.93 

.40 

.84 

Supervisors 

84-85 

8 

.59 

.92 

.57 

.92 

.57 

.92 

.59 

.92 

63B 

59-62 

6 

.57 

.89 

.62 

.91 

.46 

.84 

Incumbents 

13-14 

6 

.59 

.90 

.55 

.88 

.49 

.85 

Supervisors 

46-48 

6 

.57 

.89 

.64 

.91 

.46 

.84 

88M 

87-90 

6 

.51 

.86 

.67 

.92 

.64 

.91 

Incumbents 

16 

6 

.29 

.71 

.82 

.97 

.50 

.86 

Supervisors 

70-73 

6 

.55 

.88 

.42 

.87 

.60 

.90 

.67 

.93 

Note.  Ns  are  reported  as  ranges  because  Mean  and  SD  information  is  reported  as  average  of  individual  items.  MOS 
analyzed  in  the  Army  0*NET  study  were  19K,  25U,  3  IB,  and  88M. 


In  comparison  to  the  ratings  of  the  work  activities,  the  work  activities  categories  had 
lower  multi-rater  reliability  estimates,  but  higher  single-rater  reliability  estimates.  This  is  likely 
due  to  the  greater  number  of  items  rated  in  the  complete  work  activities  questionnaires — item 
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totals  were  reduced  by  as  much  as  80%  in  creating  the  work  activities  categories.  Reductions  in 
the  number  of  elements  included  in  the  analysis  influences  the  multi-rater  reliability  estimates, 
but  does  not  influence  the  single-rater  reliability  estimates. 

The  work  context  reliability  estimates  are  presented  in  Table  20.  All  multi-rater  reliability 
estimates  were  above  the  minimum  target  of  .80.  Multi-rater  reliability  estimates  ranged  from  .88 
to  .98.  In  general,  there  were  no  systematic  differences  in  the  reliability  estimates  across  MOS 
for  both  types  of  context  ratings;  these  estimates  ranged  from  .91  to  .96.  For  the  most  part,  there 
were  also  no  systematic  differences  in  multi-rater  reliability  estimates  between  rater  types  across 
all  category  ratings.  However,  incumbents  typically  rated  the  items  in  both  combat  and  non¬ 
combat  environments  somewhat  more  reliably  than  did  supervisors. 


Table  20.  ICCs  of  Work  Context  Ratings  by  MOS  and  Rater  Type 


MOS/Rater 

Type 

N 

Number  of 
Statements 

Non-Combat 

Ratings 

Army 

0*NET 

ICC  ICC 
(C,1)  (C,k) 

Combat  Ratings 

ICC  (C,l) 

ICC 

(C,k) 

ICC 

(C,l) 

ICC 

(C,k) 

1  IB 

69-71 

46 

.30 

.95 

.18 

.91 

Incumbents 

12-13 

46 

.48 

.98 

.21 

.93 

Supervisors 

57-58 

46 

.24 

.93 

.17 

.90 

19K 

80-84 

46 

.17 

.91 

.19 

.92 

Incumbents 

6-7 

46 

.34 

.96 

.45 

.97 

Supervisors 

73-76 

46 

.14 

.88 

.64 

.91 

.14 

.88 

25U 

244-247 

46 

.25 

.94 

.30 

.95 

Incumbents 

54-56 

46 

.32 

.96 

.38 

.97 

Supervisors 

188-189 

46 

.22 

.93 

.46 

.88 

.28 

.95 

31B 

98-100 

46 

.27 

.94 

.33 

.96 

Incumbents 

17-18 

46 

.24 

.94 

.50 

.98 

Supervisors 

81-82 

46 

.27 

.95 

.50 

.90 

.24 

.94 

63B 

72-73 

46 

.23 

.93 

.28 

.95 

Incumbents 

8-9 

46 

.20 

.92 

.26 

.94 

Supervisors 

64 

46 

.23 

.93 

.29 

.95 

88M 

137-144 

46 

.26 

.94 

.23 

.93 

Incumbents 

17-19 

46 

.55 

.98 

.43 

.97 

Supervisors 

120-125 

46 

.18 

.91 

.36 

.84 

.19 

.92 

Note.  Ns  are  reported  as  ranges  because  Mean  and  SD  information  is  reported  as  average  of  individual  items. 


Although  not  directly  comparable  to  the  context  ratings  found  in  the  Army  0*NET 
report,  we  found  higher  supervisor  multi-rater  ICCs  for  work  context  ratings  (both  non-combat 
and  combat)  for  three  of  the  MOS;  25U,  3  IB,  and  88M.  The  ICC  (C ,k)  value  for  19K  was  higher 
than  the  ICC  (C ,k)  value  in  the  Army  0*NET  study  for  both  combat  and  non-combat  ratings  of 
work  context.  In  contrast  to  the  multi-rater  reliability  comparison,  the  ICC  (C,l)  values  were 
higher  for  both  non-combat  and  combat  context  ratings  in  each  of  the  respective  MOS  in  the 
Army  0*NET  study.  The  generally  higher  values  for  ICC  (C ,k)  in  the  present  research  may  be 
due  to  the  larger  number  of  raters  and  descriptors;  the  Army  0*NET  project  had  eight  raters  per 
MOS,  whereas  the  supervisor  sample  size  for  each  MOS  in  this  research  ranged  from  33-115.  In 
sum,  the  separation  of  work  content  ratings  into  combat  and  non-combat  ratings  appears  to  have 
resulted  in  more  reliable  ratings  in  this  study  in  comparison  to  the  Army  0*NET  study. 
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Worker-Oriented  Descriptor  Reliability  Estimates 

The  reliability  estimates  for  the  worker-oriented  descriptors  are  presented  in  Tables  21 
and  22.  For  all  worker-oriented  descriptor  reliabilities,  we  computed  single  and  multi-rater 
reliability  estimates  for  each  domain:  work  abilities,  styles,  values,  and  interests.  Again,  in 
examining  the  multi-rater  reliability  estimates,  we  focus  on  comparing  the  estimates  to  the 
0*NET  minimum  target  level  of  .80  (McCloy  et  ah,  1998).  It  was  not  possible  to  make 
comparisons  between  the  work  styles,  values,  interests  descriptors  and  those  examined  in  the 
Army  0*NET  project  because  ratings  on  those  domains  were  not  collected  in  that  evaluation. 

The  work  abilities  reliability  estimates  are  presented  in  Table  21.  Multi-rater  reliability 
estimates  ranged  from  .93  to  .98,  well  above  the  0*NET  standard.  In  general,  there  were  no 
systematic  differences  in  reliability  estimates  between  MOS  across  activities  categories  ratings; 
these  estimates  ranged  from  .93  to  .97.  There  were  also  no  systematic  differences  in  multi-rater 
reliability  estimates  between  rater  types  across  abilities  ratings.  In  fact,  two  MOS  (1  IB  and  25U) 
had  identical  reliability  estimates  for  both  incumbent  and  supervisor  ratings. 


Table  21.  ICCs  of  Work  Abilities  and  Styles  Ratings  by  MOS  and  Rater  Type 


MOS/Rater 

Type 

N 

Number 

of 

Abilities 

Work  Abilities 

Army  0*NET 

ICC  ICC 

(C  ,k)  (C,k) 

N 

Number 
of  Styles 

Work  Styles 

ICC 

(C,l) 

ICC 

(C,k) 

ICC 

(C,l) 

ICC 
(C  ,k) 

1  IB 

62 

30 

.32 

.93 

55 

20 

.36 

.92 

Incumbents 

10 

30 

.31 

.93 

8 

20 

.26 

.87 

Supervisors 

52 

30 

.32 

.93 

47 

20 

.34 

.91 

19K 

56 

30 

.35 

.94 

47 

20 

.36 

.92 

Incumbents 

6 

30 

.43 

.96 

4 

20 

.15 

.78 

Supervisors 

49 

30 

.34 

.94 

.26 

.67 

42 

20 

.38 

.92 

25U 

157 

30 

.38 

.95 

136 

20 

.43 

.94 

Incumbents 

40 

30 

.41 

.95 

37 

20 

.49 

.95 

Supervisors 

117 

30 

.37 

.95 

.20 

.69 

99 

20 

.37 

.92 

31B 

100 

30 

.41 

.95 

93 

20 

.52 

.96 

Incumbents 

20 

30 

.44 

.96 

16 

20 

.58 

.96 

Supervisors 

79 

30 

.40 

.95 

.43 

.87 

76 

20 

.51 

.95 

63B 

76 

30 

.50 

.97 

65 

20 

.61 

.97 

Incumbents 

9 

30 

.63 

.98 

8 

20 

.72 

.98 

Supervisors 

67 

30 

.47 

.96 

57 

20 

.57 

.96 

88M 

89 

30 

.48 

.97 

82 

20 

.43 

.94 

Incumbents 

19 

30 

.62 

.98 

17 

20 

.70 

.98 

Supervisors 

69 

30 

.44 

.96 

.26 

.76 

64 

20 

.34 

.91 

We  generally  found  much  higher  multi-rater  ICC  estimates  for  worker  abilities  in  this 
research  than  in  the  Army  0*NET  evaluation  for  all  MOS  common  to  both  projects  except  for 
3  IB.  With  the  exception  of  3  IB,  the  ICC  (C,  1)  values  were  also  generally  higher  for  ability 
ratings  in  each  of  the  respective  MOS  in  this  study.  Although  it  appears  that  the  participants  in 
the  present  research  made  more  reliable  ratings  of  the  abilities  items,  it  is  important  to  remember 
the  large  differences  in  sample  size  between  this  effort  and  the  Anny  0*NET  study. 
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The  work  styles  reliability  estimates  are  also  presented  in  Table  21.  In  general,  all  multi¬ 
rater  reliability  estimates  were  above  the  0*NET  standard  (see  the  19K  incumbent  ratings  for  an 
exception).  Multi-rater  reliability  estimates  ranged  from  .78  to  .98.  There  were  no  systematic 
differences  in  reliability  estimates  between  MOS.  In  contrast,  there  were  some  differences  in 
multi-rater  reliability  estimates  between  rater  types.  For  instance,  1  IB  and  19K  supervisors  rated 
styles  somewhat  more  reliably  than  did  incumbents  in  these  MOS. 

The  work  values  reliability  estimates  are  presented  in  Table  22.  In  general,  all  multi-rater 
reliability  estimates  were  sufficiently  high,  ranging  from  .92  to  .97.  There  were  no  systematic 
differences  in  reliability  estimates  between  MOS  across  ratings,  which  ranged  from  .94  to  .97. 
Likewise,  there  were  no  systematic  differences  between  multi-rater  reliability  estimates  between 
rater  types  across  these  ratings. 

The  work  interests  reliability  estimates  presented  in  Table  22  range  from  .94  to  .97.  There 
were  no  systematic  differences  in  reliability  estimates  between  MOS  or  rater  types. 


Table  22.  ICCs  of  Work  Values  and  Interests  Ratings  by  MOS  and  Rater  Type 


MOS/Rater 

Type 

N 

Number  of 
Values 

Work  Values 

N 

Number  of 
Interests 

Work  Interests 

ICC 

(C,l) 

ICC 
(C  ,k) 

ICC 

(C,l) 

ICC 

(C,k) 

1  IB 

48 

30 

.38 

.95 

56 

30 

.42 

.96 

Incumbents 

7 

30 

.40 

.95 

8 

30 

.46 

.96 

Supervisors 

41 

30 

.36 

.94 

48 

30 

.41 

.95 

19K 

47 

30 

.36 

.94 

52 

30 

.37 

.95 

Incumbents 

7 

30 

.45 

.96 

10 

30 

.35 

.94 

Supervisors 

40 

30 

.35 

.94 

42 

30 

.37 

.95 

25U 

168 

30 

.29 

.93 

180 

30 

.35 

.94 

Incumbents 

42 

30 

.36 

.94 

44 

30 

.39 

.95 

Supervisors 

126 

30 

.27 

.92 

136 

30 

.34 

.94 

31B 

74 

30 

.46 

.96 

93 

30 

.44 

.96 

Incumbents 

8 

30 

.47 

.96 

13 

30 

.54 

.97 

Supervisors 

66 

30 

.43 

.96 

80 

30 

.43 

.96 

63B 

42 

30 

.38 

.95 

52 

30 

.41 

.95 

Incumbents 

10 

30 

.31 

.93 

14 

30 

.34 

.94 

Supervisors 

32 

30 

.40 

.95 

38 

30 

.44 

.96 

88M 

100 

30 

.49 

.97 

113 

30 

.51 

.97 

Incumbents 

17 

30 

.40 

.95 

18 

30 

.50 

.97 

Supervisors 

83 

30 

.52 

.97 

95 

30 

.52 

.97 
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Summary  of  the  Inter-Rater  Reliability  Analyses 

In  general,  we  found  that  both  the  work-oriented  and  worker-oriented  descriptors  were 
rated  reliably  by  both  incumbents  and  supervisors  both  within  and  across  MOS.  Very  few 
descriptors  demonstrated  ICC  (C ,k)  values  less  than  the  0*NET  .80  standard  (McCloy  et  al, 
1998).  However,  the  ICC  (C,l)  values  were  not  quite  as  high  as  the  values  observed  in  the  Anny 
0*NET  Evaluation.  The  single-rater  reliability  differences  were  most  likely  due  to  rater  training 
and  proctoring  in  the  Army  0*NET  study,  but  absent  in  the  present  research.  The  general  pattern 
was  that  the  worker-oriented  domain  descriptors  were  rated  slightly  more  reliably  than  the  work- 
oriented  domain  descriptors. 

Did  Incumbents  and  Supervisors  Agree  in  their  Ratings? 

Three  statistics  are  reported  in  the  Tables  23  to  26:  (a)  profile  similarity  (r)  in  item 
ratings,  (b)  mean  of  the  standardized  mean  item  differences  (or  d),  and  (c)  the  average  absolute  d 
(or  \d\)  on  the  items.  The  profile  similarity  (r)  reflects  the  agreement  between  incumbent  and 
supervisor  ratings  within  each  domain,  and  was  computed  by  taking  the  average  correlation 
between  the  raters’  ratings  of  descriptors  within  each  domain.  Higher  values  of  r  indicate  a 
higher  degree  of  similarity  in  the  shape  of  the  profiles  between  incumbent  and  supervisor  ratings 
of  the  items  in  each  domain.  The  mean  of  the  standardized  mean  item  differences  (or  d) 
represents  the  difference  in  magnitude  between  incumbent  and  supervisor  ratings  of  the  items 
comprising  a  domain.  The  standardized  mean  differences  were  computed  by  calculating  the 
difference  between  the  mean  incumbent  and  supervisor  ratings  of  each  item  within  each  domain, 
standardizing  the  difference  (i.e.,  dividing  the  difference  by  the  pooled  standard  deviation  of 
each  item),  and  taking  the  average  difference  across  items  within  the  domain.  Positive  values 
mean  that  the  average  ratings  made  by  supervisors  were  higher  than  those  made  by  incumbents. 
Large  values,  positive  or  negative,  indicate  less  agreement  in  the  mean  ratings  of  incumbents  and 
supervisors.  Finally,  we  calculated  the  average  absolute  d.  This  value  was  calculated  by  taking 
the  absolute  difference  between  the  mean  incumbent  and  supervisor  ratings  of  each  item  within 
each  domain.  By  taking  the  absolute  value  of  d,  we  were  able  to  examine  the  pure  difference 
between  the  incumbent  and  supervisor  ratings  within  each  MOS  and  each  domain  without 
concern  of  directionality  and  its  influence  on  the  mean  difference  calculation. 

Work-Oriented  Descriptor  Similarities  and  Mean  Rater  Differences 

The  work-oriented  similarities  and  mean  rater  differences  are  reported  in  Tables  23  to  25. 
Table  23  reports  the  profile  similarity  and  mean  rater  differences  for  the  work  activities  domain 
by  scale,  broken  out  by  MOS  and  rater  type.  The  similarity  indices  ranged  from  .33  to  .88  for  all 
three  types  of  activity  ratings.  Incumbents  and  supervisors  tended  to  agree  on  the  trainability  (r  = 
.82)  of  the  activities  comprising  the  domain  to  a  greater  extent  than  they  did  on  importance  and 
criticality,  but  the  mean  ratings  of  importance  and  criticality  also  reflected  a  high  degree  of 
similarity  (r  =  .73  for  both  importance  and  criticality).  Across  MOS,  there  were  some  differences 
in  profile  similarities  for  importance,  trainability,  and  criticality.  In  particular,  the  similarity 
index  for  25U  was  far  lower  than  the  other  MOS  for  ratings  of  importance  and  criticality,  but  the 
index  for  the  trainability  ratings  was  in  line  with  the  other  MOS. 
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Table  23.  Profile  Similarity  and  Mean  Differences  in  Activities  Ratings  between  Incumbents 
and  Supervisors 


Mean  Differences 

Domain/MOS 

r 

Mean  of  d 

Average 
Absolute  d 

Work  Activities  (Importance) 

.73 

-.11 

21 

1  IB 

.82 

-.05 

.20 

19K 

.78 

-.50 

.51 

25U 

.33 

.14 

.27 

31B 

.81 

-.23 

.25 

63B 

.76 

-.24 

.26 

88M 

.78 

.18 

.22 

Work  Activities  (Trainability) 

.82 

.26 

.30 

1  IB 

.87 

.32 

.32 

19K 

.75 

.20 

.31 

25U 

.79 

.10 

.15 

31B 

.85 

.20 

.22 

63B 

.77 

.05 

.19 

88M 

.81 

.64 

.65 

Work  Activities  (Criticality) 

.73 

-.06 

.25 

1  IB 

.83 

-.08 

.22 

19K 

.77 

-.32 

.34 

25U 

.43 

.17 

.27 

3  IB 

.74 

-.18 

.25 

63B 

.76 

-.06 

.18 

88M 

.78 

.25 

.29 

Note.  Positive  ds  indicate  that  the  average  ratings  made  by  supervisors  were  higher  than  those  made  by  incumbents. 

The  standardized  mean  differences  between  incumbent  and  supervisor  ratings  of 
activities,  represented  by  mean  ds,  ranged  between  -.50  to  .64  across  the  three  types  of  activities 
ratings.  The  mean  d  for  importance  and  criticality  ratings  indicated  that  incumbents  generally 
made  slightly  higher  ratings  (-.11  and  -.06,  respectively)  relative  to  supervisors.  In  contrast,  the 
mean  d  for  trainability  ratings  indicated  that  supervisors  generally  rated  items  as  more  difficult  to 
learn  than  did  incumbents  (.26). 


In  comparing  MOS-specific  values  to  the  sample-weighted  mean  d  for  each  type  of 
activity  rating,  divergence  from  the  mean  d  exists  for  some  MOS.  For  importance  ratings,  the  d 
value  for  19K  indicated  that  the  incumbents  generally  rated  the  activities  as  more  important  than 
the  supervisors  rated  these  same  activities  (-.50).  However,  this  difference  is  markedly  larger 
than  the  sample -weighted  mean  d  for  importance  ratings  (d=  1 1).  For  trainability  ratings,  63B 
and  88M  diverged  from  the  mean  d.  While  the  d  value  for  63B  indicated  that  incumbents  and 
supervisors  were  more  in  agreement  (e.g.,  .05  compared  to  .26)  on  their  ratings  of  trainability 
than  the  average  across  MOS,  88M  indicated  a  far  greater  difference  (e.g.,  .64  compared  to  .26) 
between  incumbent  and  supervisor  ratings  of  trainability.  Finally,  for  criticality  ratings,  19K  and 
88M  diverged  from  the  mean  d.  The  supervisors  and  incumbents  in  19K  differed  to  a  larger 
extent  on  level  of  criticality  than  the  sample- weighted  mean  d,  with  incumbents  in  19K  tending 
to  rate  the  activities  as  more  critical  (just  as  they  had  rated  them  as  more  important)  than  the 
supervisors.  In  contrast,  although  the  supervisors  and  incumbents  in  88M  also  reflected  a  high 
degree  of  similarity  on  ratings  of  criticality,  supervisors  in  88M  tended  to  make  higher  ratings  of 
trainability  than  did  incumbents.  It  should  be  noted  that  across  MOS,  supervisors  consistently 
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rated  the  activities  comprising  the  domain  as  more  difficult  to  train.  This  type  of  pattern  did  not 
occur  for  ratings  of  importance  or  criticality. 

The  absolute  mean  differences  (\d\)  between  incumbent  and  supervisor  ratings  of 
activities  across  MOS  ranged  from  .18  to  .65.  The  average  absolute  ds  were  .27,  .30,  and  .25, 
respectively,  for  ratings  of  importance,  trainability,  and  criticality.  Despite  the  fact  that 
supervisors  may  have  rated  the  activities  as  more  important,  trainable,  or  difficult  than  the 
incumbents  (or  the  converse),  the  absolute  values  of  these  differences  were  not  typically  very 
large. 


There  was  some  divergence  across  MOS.  For  instance,  raters  in  19K  generally  differed  to 
a  larger  extent  on  the  level  of  importance  of  activities  than  the  other  MOS.  Likewise,  raters  in 
88M  differed  in  their  ratings  of  level  of  trainability  of  activities  to  a  larger  extent  than  the  other 


MOS. 


Table  24  reports  the  profile  similarity  and  mean  differences  for  the  work  activities 
categories  by  MOS  and  rater  type.  The  similarity  indices,  represented  by  correlations,  ranged 
between  -.29  and  .93  across  all  three  ratings.  Incumbents  and  supervisors  tended  to  agree  to  a 
greater  extent  on  the  trainability  (r  =  .68)  of  the  activities  categories  than  they  did  on  the 
importance  and  criticality  (r  =  .53  and  r  =  .55,  respectively),  but  the  mean  ratings  of  importance 
and  criticality  also  reflected  a  moderate  degree  of  similarity.  Across  MOS,  there  were  some  large 
differences  in  profile  similarities  for  importance,  trainability,  and  criticality.  For  instance,  the 
correlations  between  incumbent  and  supervisor  ratings  of  importance  for  1  IB  (r  =  .92)  and  88M 
(r  =  .85)  were  much  larger  than  the  sample-weighted  mean  correlation  for  importance  ratings 
across  MOS,  whereas  the  correlation  for  importance  ratings  for  63B  was  much  smaller  (r  =  .  13). 
Similarly,  the  correlations  for  trainability  (r  =.05)  and  criticality  (r  =  -.29)  ratings  were  much 
lower  for  63B  than  the  sample-weighted  mean  d.  It  is  important  to  note  that  despite  the  lack  of  a 
large  mean  difference  ( d  =  -.20)  in  the  average  criticality  ratings  for  63B,  the  rank  orders  of  the 
items  by  incumbents  and  supervisors  directly  contributed  to  the  negative  correlation.  In  contrast, 
the  correlations  between  incumbent-supervisor  criticality  ratings  for  1  IB  (r  =  .93)  and  88M  (r  = 
.91)  were  much  larger  than  the  sample- weighted  mean  similarity  index. 

The  standardized  mean  differences,  represented  by  mean  ds,  ranged  between  -.32  to  .90. 
The  mean  d  for  importance  ratings  indicated  that  incumbents  generally  made  slightly  higher 
ratings  (-.12),  as  did  the  mean  d  for  criticality  ratings  (-.08).  In  contrast,  supervisors  generally 
rated  items  as  more  difficult  to  train  than  did  incumbents  (mean  d  =  .36). 

For  importance  ratings,  the  d  value  for  19K  indicated  that  the  incumbents  generally  rated 
the  activities  comprising  the  domain  as  more  important  than  the  supervisors  rated  these  same 
activities  (-.32).  For  trainability  ratings,  88M  diverged  from  the  mean  d  such  that  there  was  a 
greater  difference  (i.e.,  .90  compared  to  .36)  between  incumbent  and  supervisor  ratings  of 
trainability  than  the  sample-weighted  mean  d.  It  is  important  to  note  that  despite  the  high  degree 
of  similarity  between  incumbent  and  supervisor  trainability  ratings  (r  =  .79),  there  was  still  a 
large  difference  in  magnitude  between  the  incumbent  and  supervisor  average  ratings. 
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Table  24.  Profile  Similarity  and  Mean  Differences  in  Work  Activities  Categories  Ratings 
between  Incumbents  and  Supervisors 


Mean  Differences 

Domain/MOS 

r 

Mean  of  d 

Average  Absolute  d 

Work  Activities-Categories  (Importance) 

.53 

-.12 

.25 

1  IB 

.92 

-.05 

.16 

19K 

.47 

-.32 

.38 

25U 

.78 

.15 

.19 

3  IB 

.37 

-.28 

.28 

63B 

.13 

-.29 

.30 

88M 

.85 

.15 

.18 

Work  Activities-Categories  (Trainability) 

.68 

.36 

.40 

1  IB 

.73 

.50 

.50 

19K 

.71 

.51 

.51 

25U 

.81 

-.02 

.11 

31B 

.82 

-.02 

.08 

63B 

.05 

.25 

.29 

88M 

.79 

.90 

.90 

Work  Activities-Categories  (Criticality) 

.55 

-.08 

.27 

1  IB 

.93 

-.13 

.23 

19K 

.27 

-.24 

.36 

25U 

.79 

.15 

.20 

31B 

.42 

-.20 

.20 

63B 

-.29 

-.20 

.33 

88M 

.91 

.34 

.34 

Note.  Positive  ds  indicate  that  the  average  ratings  made  by  supervisors  were  higher  than  those  made  by  incumbents. 


The  absolute  mean  differences  (|r/|)  between  incumbent  and  supervisor  ratings  of 
activities  categories  across  MOS  ranged  from  .08  to  .90.  The  average  absolute  ds  were  .25,  .40, 
and  .27,  respectively,  for  the  importance,  trainability,  and  criticality  ratings.  There  was  some 
divergence  across  MOS.  For  instance,  raters  in  88M  generally  differed  to  a  larger  extent  on  the 
perceived  levels  of  activity  category  trainability  than  the  sample-weighted  mean  \d\.  However, 
within  the  other  activities  categories  ratings,  there  were  more  uniform  absolute  differences  in  the 
ratings  of  incumbents  and  supervisors.  The  absolute  differences  in  the  ratings  of  trainability  for 
the  activities  categories  were  the  largest  of  the  three  scales. 

Table  25  reports  the  profile  similarity  and  mean  differences  for  the  work  context  ratings 
by  MOS,  rater  type,  and  environment.  The  similarity  indices  ranged  from  .58  to  .95  for  both 
types  of  context  ratings  (combat  vs.  non-combat).  Incumbents  and  supervisors  tended  to  agree  to 
a  greater  extent  on  the  work  context  ratings  in  combat  environments  (r  =  .86)  than  they  did  on 
the  work  context  ratings  in  non-combat  environments  (r  =  .78),  but  the  ratings  for  both  types  of 
environments  reflect  a  high  degree  of  similarity.  Across  MOS,  there  were  very  few  differences 
between  the  profile  similarities  for  context  ratings,  especially  within  the  ratings  of  work  context 
in  combat  environments  (i.e.,  the  correlations  ranged  from  .80  to  .92,  indicating  that  supervisors 
and  incumbents  tend  to  view  the  context  of  the  work  in  a  similar  fashion  in  combat 
environments).  However,  the  correlations  between  incumbent  and  supervisor  ratings  of  context 
in  non-combat  for  19K  (r  =  .58)  and  63B  (r  =  .68)  were  somewhat  lower  than  the  sample- 
weighted  mean  correlation  across  MOS  (r  =  .78).  Nevertheless,  both  of  these  values  reflect  at 
least  moderate  levels  of  similarity  between  ratings  of  the  work  context  in  non-combat 
environments  by  supervisors  and  incumbents. 
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The  standardized  mean  differences  (mean  ds),  ranged  between  -.36  to  .61.  They  show  that 
incumbents  generally  made  slightly  higher  non-combat  environment  ratings  (-.10),  but  that 
supervisor  ratings  in  combat  enviromnents  (.15)  were  slightly  higher  than  ratings  made  by 
incumbents. 


There  were  few  differences  between  the  MOS-specific  ds  and  the  sample-weighted  mean 
d  for  the  context  ratings  in  non-combat  environments.  Similar  to  the  work  activities  ratings,  the 
directionality  of  differences  (i.e.,  whether  supervisors’  or  incumbents’  ratings  were  higher) 
varied  across  MOS.  However,  the  range  of  differences  between  supervisor  and  incumbent  ratings 
was  generally  small  in  non-combat  environments.  Similarly,  the  differences  between  supervisor 
and  incumbent  ratings  of  context  in  combat  environments  were  relatively  uniform  in  magnitude 
but  differed  in  directionality.  The  exception  to  this  pattern  was  3  IB  in  which  supervisors  rated 
the  work  context  items  as  being  more  important  in  a  combat  environment  than  incumbents. 

The  absolute  mean  differences  (|r/|)  between  incumbent  and  supervisor  ratings  of  work 
context  in  both  combat  and  non-combat  environments  across  MOS  ranged  from  .17  to  .61.  The 
average  absolute  ds  were  .32  and  .34,  respectively,  for  ratings  of  work  context  in  both  types  of 
environment.  There  was  relatively  little  divergence  across  MOS,  although  the  \d\  for  incumbent 
and  supervisor  ratings  in  19K  was  somewhat  higher  than  the  sample -weighted  mean  \d\.  In 
contrast  to  the  relatively  small  degree  of  variation  on  \d\  in  non-combat  ratings,  combat 
environment  context  ratings  demonstrated  a  larger  degree  of  variation  across  MOS.  For  instance, 
raters  in  3  IB  differed  to  a  larger  extent  on  the  degree  to  which  the  work  context  items  were 
characteristic  in  a  combat  environment  to  a  much  greater  degree  than  the  sample-weighted  mean 
\d\.  In  contrast,  the  average  context  ratings  in  non-combat  environments  of  supervisors  and 
incumbents  in  25U  reflected  a  smaller  degree  of  difference  than  the  sample-weighted  mean  \d\. 


Table  25.  Profile  Similarity  and  Mean  Differences  in  Work  Context  Ratings  between 
Incumbents  and  Supervisors 


Mean 

Differences 

Domain/MOS 

r 

Mean  of  d 

Average  Absolute  d 

Work  Context  (Non-Combat) 

.78 

-.10 

.32 

1  IB 

.80 

-.28 

.35 

19K 

.58 

-.36 

.56 

25U 

.95 

.02 

.17 

3  IB 

.88 

.22 

.27 

63B 

.68 

.06 

.30 

88M 

.80 

-.24 

.31 

Work  Context  (Combat) 

.86 

.15 

.34 

1  IB 

.89 

-.21 

.34 

19K 

.80 

.34 

.46 

25U 

.92 

.15 

.19 

31B 

.84 

.61 

.61 

63B 

.82 

.05 

.26 

88M 

.89 

-.06 

.20 

Note.  Positive  ds  indicate  that  the 

average  ratings  made  by 

supervisors  were 

higher  than  those 

made  by  incumbents. 
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Worker-Oriented  Descriptor  Similarities  and  Mean  Rater  Differences 


Table  26  reports  the  profile  similarity  and  mean  differences  for  the  worker-oriented  domains. 
The  similarity  indices  ranged  from  .23  to  .93  across  all  domains.  Incumbents  and  supervisors  tended 
to  agree  to  a  greater  extent  on  the  ratings  of  interests  (r  =.86)  than  on  the  ratings  of  abilities,  styles, 
and  values  (r  =  .76,  r  =  .61,  and  r  =  .70,  respectively),  but  the  correlations  for  abilities,  styles,  and 
values  all  reflected  a  large  degree  of  similarity  between  incumbent  and  supervisor  ratings.  Across 
worker-oriented  domains  and  MOS,  there  were  some  differences  in  profile  similarity  indices  in 
comparison  to  the  sample-weighted  mean  <7s.  For  instance,  the  correlation  between  incumbent  and 
supervisor  ratings  of  abilities  for  1  IB  (r  =.51)  was  somewhat  lower  than  the  sample-weighted  mean 
correlation  for  importance  ratings  across  MOS.  Similarly,  the  correlation  for  styles  ratings  for  63B  (r 
=  .23)  was  much  smaller  than  the  sample-weighted  mean  correlation.  The  correlations  for  values  and 
interests  were  relatively  unifonn  (and  high)  across  MOS.  In  general,  the  profile  similarity  indices  for 
worker-oriented  domains  were  high  across  MOS. 

Table  26.  Profile  Similarity  and  Mean  Differences  in  Work  Abilities,  Styles,  Values  and 
Interests  Ratings  between  Incumbents  and  Supervisors 


Mean  Differences 

Domain/MOS 

r 

Mean  of  d 

Average  Absolute  d 

Abilities 

.76 

-.07 

.29 

1  IB 

.73 

-.23 

.35 

19K 

.51 

-.36 

.48 

25U 

.95 

.18 

.20 

31B 

.86 

-.18 

.25 

63B 

.70 

.30 

.34 

88M 

.79 

-.13 

.19 

Styles 

.61 

-.10 

.42 

1  IB 

.76 

-.62 

.62 

19K 

.67 

-.51 

.68 

25U 

.81 

.36 

.39 

31B 

.53 

-.29 

.33 

63B 

.23 

.48 

.51 

88M 

.68 

-.04 

.17 

Values 

.70 

.19 

.34 

1  IB 

.76 

.55 

.55 

19K 

.66 

-.13 

.29 

25U 

.68 

.08 

.22 

31B 

.69 

.74 

.74 

63B 

.78 

-.13 

.20 

88M 

.61 

.06 

.23 

Interests 

.86 

-.05 

.24 

1  IB 

.86 

.16 

.30 

19K 

.86 

-.29 

.34 

25U 

.93 

-.08 

.15 

31B 

.85 

.11 

.27 

63B 

.88 

-.09 

.20 

88M 

.80 

-.10 

.24 

Note.  Positive  ds  indicate  that  the  average  ratings  made  by  supervisors  were  higher 
than  those  made  by  incumbents. 
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The  standardized  mean  differences  ranged  between  -.62  to  .74.  The  mean  ds  for  abilities, 
styles,  and  interests  indicated  that  incumbents  generally  made  slightly  higher  ratings  (-.07,  -.10, 
and  -.05,  respectively)  than  supervisors;  however,  the  mean  d  for  values  ratings  indicated  that 
supervisors  generally  made  slightly  higher  ratings  (.19)  than  incumbents. 

In  comparing  MOS-specific  values  to  the  sample-weighted  mean  d  for  each  type  of  worker- 
oriented  domain,  some  divergence  from  the  mean  d  shows  up  across  MOS  within  each  domain.  For 
abilities  ratings,  the  d  values  for  19K  (d  =  -.36)  and  63B  (d=  .30)  indicated  a  greater  difference  from 
the  sample-weighted  mean  than  did  the  other  MOS.  Although  the  directionality  of  these  two  d  values 
differs,  indicating  that  supervisors  in  63B  rated  the  abilities  items  more  highly  than  did  incumbents 
while  incumbents  in  19K  rated  the  abilities  items  more  highly,  the  magnitude  of  both  differences  is 
comparable  and  fairly  large  in  comparison  to  the  sample-weighted  mean  d  for  abilities  ratings.  For 
styles  ratings,  both  1  IB  and  19K  demonstrate  d  values  in  the  same  direction  as  the  sample- weighted 
mean  d,  indicating  that  incumbents  rated  the  work  styles  items  more  highly  than  did  the  supervisors 
in  these  MOS;  however,  both  values  ( d  =  -.62,  and  d  =  -.51,  respectively)  are  much  larger  than  the 
sample-weighted  mean  d  for  styles  ( d  =  -.10).  For  values  ratings,  supervisors  in  1  IB  and  in  3  IB  both 
rated  the  values  items  more  highly  than  did  the  incumbents,  which  is  in  line  with  the  sample- 
weighted  mean  d  for  values.  However,  the  d  values  in  these  MOS  ( d  =  .55,  and  d  =  .74,  respectively) 
are  much  larger  than  the  sample- weighted  mean  d  (.19). 

The  absolute  mean  differences  (|^|)  between  incumbent  and  supervisor  ratings  of  worker- 
oriented  domains  across  MOS  ranged  from  .15  to  .74.  The  average  absolute  ds  were  .29,  .42,  .34, 
and  .24  for  abilities,  styles,  values,  and  interests,  respectively.  In  comparing  MOS-specific  values 
to  the  sample- weighted  mean  |z/|  for  each  domain,  there  was  some  divergence  across  MOS. 

Raters  in  19K  (.48)  generally  differed  to  a  larger  extent  on  the  ratings  of  abilities  than  the 
sample-weighted  mean  \d\  for  abilities  (.29).  Similarly,  1  IB  (.62)  and  19K  (.68)  exhibited  much 
larger  differences  between  the  supervisor  and  incumbent  ratings  of  styles  than  the  sample- 
weighted  mean  \d\  (.42)  exhibited.  Finally,  1  IB  (.55)  and  3  IB  (.74)  also  differed  more  on 
supervisor  and  incumbent  ratings  of  values  than  the  sample-weighted  \d\  (.34)  indicated. 

Despite  these  larger  than  expected  absolute  differences,  it  is  important  to  note  that  for  a 
number  of  these  MOS,  the  similarity  indices  were  quite  high.  The  high  degree  of  similarity  and 
the  moderate  degrees  of  difference  for  1  IB  and  19K  ratings  of  styles,  and  for  1  IB  and  3  IB 
ratings  of  values,  are  not  mutually  exclusive.  For  these  MOS,  the  average  means  of  the 
supervisor  and  incumbent  ratings  of  items  were  different,  but  they  reflected  a  similar  rank 
ordering  by  item  across  rater  type. 

Summary  of  Incumbent-Supervisor  Agreement  in  Ratings 

Within  the  work-oriented  descriptors,  supervisor  and  incumbent  ratings  of  activities  and 
activities  categories  descriptors  generally  demonstrated  a  high  degree  of  similarity;  however, 
activities  categories  correlations  were  somewhat  lower  than  the  correlations  for  the  full  set  of 
work  activities  items.  In  general,  incumbent  ratings  were  higher  than  supervisors  for  ratings  of 
importance  and  criticality,  while  trainability  ratings  were  higher  for  supervisors  across  activities 
and  activities  categories.  Absolute  standardized  mean  differences  were  generally  low  across  all 
three  types  of  ratings  for  work  activities  and  categories,  indicating  a  low  level  of  difference 
between  the  mean  ratings  across  the  items  comprising  both  scales. 
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Also  within  the  work-oriented  descriptors  domain,  the  work  context  supervisor  and 
incumbent  ratings  generally  indicated  that  the  similarity  indices  were  high  for  both  work  context 
descriptors  in  a  non-combat  environment  and  in  a  combat  environment;  however,  the  similarity 
indices  were  higher  for  the  combat  environment  ratings.  In  non-combat  environments, 
incumbents  made  higher  ratings  on  average  than  supervisors;  however,  in  combat  environments, 
supervisor  ratings  were  higher.  Neither  of  these  differences  was  large  and  was  supported  by  the 
relatively  small  differences  evident  in  the  absolute  standardized  mean  differences. 

Within  the  worker-oriented  descriptors,  the  similarity  indices  were  higher  than  those  for 
the  work-oriented  descriptors,  with  the  values  indices  being  the  highest  of  the  four  domains.  For 
abilities,  styles,  and  interests,  the  incumbents  generally  rated  the  descriptors  more  highly  than  did 
incumbents;  supervisors  generally  made  higher  ratings  than  incumbents  for  values.  The  absolute 
standardized  mean  differences  for  the  worker-oriented  descriptors  were  generally  in  line  with  the 
work-oriented  descriptor  differences. 

In  summary,  we  found  relatively  high  degrees  of  similarities  and  small  differences 
between  incumbent  and  supervisor  ratings  for  both  work-oriented  and  worker-oriented  domains 
both  within  and  across  MOS. 

Did  Ratings  from  the  Questionnaires  Differentiate  among  MOS? 

Consistent  with  the  Army  0*NET  Evaluation,  we  used  a  multi-trait  multi-method 
approach  (Campbell  &  Fiske,  1959)  to  answer  this  question.  The  profile  similarities  and  mean 
differences  among  MOS  are  presented  in  Tables  27  and  28.  The  numbers  below  the  diagonal  are 
cross-MOS  correlations  in  the  profile  of  ratings  on  the  targeted  domain  (e.g.,  how  did  the  ability 
ratings  for  1  IB  correlate  with  those  of  19K?  Or  3  IB?  Or  63B?).  The  numbers  above  the  diagonal 
reflect  the  average  absolute  mean  standardized  difference  (or  d)  in  the  ratings  between  MOS  A 
versus  those  of  MOS  B  (i.e.,  how  much  did  the  ability  ratings  for  1  IB  differ  from  those  of  19K, 
on  average,  in  standard  deviation?). 

The  numbers  on  the  diagonal  correspond  to  the  two  aforementioned  metrics  but  capture 
that  infonnation  within  an  MOS,  specifically  between  the  two  rater  types  (incumbents  versus 
supervisors).  The  number  before  the  "/"  represents  the  correlation  between  the  incumbents'  and 
supervisors'  ratings  on  the  domain  in  question  for  the  selected  MOS,  whereas  the  number  after 
the  "/"  reflects  the  average  absolute  mean  standardized  difference  (or  d)  between  the  incumbent 
and  supervisor  ratings  for  the  same  MOS. 

Consistent  with  a  multi-trait  multi-method  framework,  high  profile  similarities  (or  low 
mean  differences)  among  different  rater  types  for  the  same  MOS  would  indicate  convergent 
validity.  That  is,  different  methods  (i.e.,  rater  types)  yield  converging  results  for  the  same  MOS. 
If  these  convergent  validity  estimates  are  higher  than  the  corresponding  estimates  between 
different  MOS,  there  is  evidence  of  discriminant  or  divergent  validity.  Divergent  validity  would 
indicate  that  the  descriptors  are  differentiating  across  MOS.  In  sum,  it  would  be  desirable  if  (a) 
the  differences  in  the  ratings  between  the  two  rater  types  within  an  MOS  were  smaller  and  their 
similarity  higher  than  the  ratings  across  different  MOS  and  (b)  the  pattern  of  similarity  and 
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differences  in  the  ratings  across  different  MOS  followed  the  pattern  one  would  expect.  As  an 
example  of  the  latter,  those  MOS  that  are  more  similar  to  each  other  in  their  job  requirements, 
say  1  IB  and  19K,  would  show  greater  similarity  and  smaller  differences  in  their  ratings  than 
those  MOS  that  were  more  dissimilar,  say  1  IB  and  25U. 

MOS  Differentiation  with  Work-Oriented  Descriptors 

Table  27  presents  the  similarity  and  mean  differences  in  work  context  ratings  in  non-combat 
environments  among  the  MOS.  The  results  for  the  work  context  ratings  in  non-combat  environments 
generally  reflected  correlations  between  MOS  that  were  comparable  to  the  correlations  within  MOS 
across  rater  types.  For  context  ratings  in  non-combat  enviromnents,  the  within  MOS  average 
correlation  was  .78,  while  the  between  MOS  average  correlation  was  .66.  Similarly,  for  combat 
enviromnent  ratings,  the  average  correlation  within  MOS  was  .86,  while  the  between  MOS 
correlation  was  .77  (see  Table  27).  For  some  MOS  combinations,  however,  the  correlations  between 
the  MOS  were  greater  than  the  correlations  between  incumbent  and  supervisor  ratings  within  the 
MOS.  For  instance,  the  similarities  for  work  context  ratings  in  a  non-combat  enviromnent  between 
1  IB,  19K,  63B,  and  88M  were  stronger  than  the  within-MOS  correlations  between  raters  in  these 
MOS.  Likewise,  the  difference  scores  between  similar  MOS  (e.g.,  1  IB  and  19K)  were  generally 
smaller  than  the  difference  scores  between  incumbent  and  supervisor  ratings  within  both  of  these 
MOS.  Similarly,  the  relationship  between  63B  and  88M  resulted  in  a  smaller  difference  score  than 
the  difference  score  for  incumbent  and  supervisor  ratings  within  both  of  these  MOS.  Finally, 
dissimilar  MOS  demonstrated  small  similarity  indices  in  non-combat  enviromnents  (e.g.,  the 
correlations  between  19K  and  3  IB  and  between  25U  and  3  IB  were  small). 


Table  27.  Profile  Similarity  and  Mean  Differences  in  Work  Context  Ratings  among  MOS 


1  IB 

19K 

25U 

31B 

63B 

88M 

Work  Context 

-Non-Combat 

1  IB 

.80/35 

.20 

.37 

.35 

.25 

.26 

19K 

.86 

.58/56 

.42 

.45 

.21 

.27 

25U 

.57 

.55 

.95/17 

.49 

.27 

.29 

31B 

.53 

.40 

.45 

.88/27 

.43 

.48 

63B 

.75 

.87 

.72 

.39 

.68/30 

.20 

88M 

.83 

.88 

.72 

.49 

.88 

.80/31 

Work  Context 

-Combat 

1  IB 

.89/34 

.15 

.51 

.25 

.43 

.26 

19K 

.96 

.80/46 

.50 

.26 

.40 

.22 

25U 

.52 

.50 

.92/19 

.40 

.21 

.35 

31B 

.93 

.89 

.45 

.84/61 

.32 

.22 

63B 

.78 

.79 

.81 

.71 

.82/26 

.23 

88M 

.91 

.92 

.65 

.86 

.89 

.89/20 

Table  27  also  presents  the  similarity  indices  and  mean  differences  in  work  context  ratings 
in  combat  environments.  The  results  for  the  work  context  ratings  in  combat  environments 
generally  reflected  correlations  between  MOS  that  were  comparable  to  the  correlations  within 
MOS  across  rater  types.  For  some  MOS  combinations,  however,  the  correlations  between  the 
MOS  were  far  greater  than  the  correlations  between  incumbent  and  supervisor  ratings  within 
multiple  MOS.  For  instance,  the  similarities  for  work  context  ratings  in  a  combat  environment 
between  1  IB  and  19K,  1  IB,  3  IB,  1  IB  and  88M,  were  generally  stronger  than  the  within-MOS 
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correlations  between  raters  for  these  MOS.  Likewise,  the  difference  scores  between  similar  MOS 
(e.g.,  1  IB  and  19K)  were  generally  smaller  than  the  difference  scores  between  incumbent  and 
supervisor  ratings  within  both  of  these  MOS.  However,  the  correlations  among  and  between 
dissimilar  MOS  in  combat  environments  were  far  greater  than  those  in  non-combat  environments 
(e.g.,  the  correlation  between  19K  and  3  IB  goes  from  .40  to  .89  when  the  context  items  are  rated 
in  a  combat  environment).  The  similarity  indices  indicate  that  the  introduction  of  the  combat 
environment  decreases  the  degree  of  discriminant  validity  across  MOS. 

Table  28  provides  a  high  level  summary  of  the  data  presented  in  Table  27.  Overall, 
supervisors  and  incumbents  rated  the  work  context  of  their  MOS  similarly,  on  average,  across  both 
non-combat  and  combat  based  enviromnents.  As  expected,  the  work  context  ratings  in  a  combat 
enviromnent  were  more  similar  across  MOS  than  those  context  ratings  in  a  non-combat  enviromnent. 


Table  28.  Average  Profile  Similarity  and  Mean  Difference  Statistics  for  the  Work-Oriented 
Descriptors 


Work  Activities 

Work  Activities  Categories 

Work  Context 

Importance 

Training  Criticality 

Importance 

Training 

Criticality 

Non- 

Combat 

Combat 

Correlations 

By  MOS 

~ 

- 

- 

- 

- 

0.66 

0.77 

By  Rater 

Type 

0.73 

0.82  0.73 

0.53 

0.68 

0.55 

0.78 

0.86 

Mean  Differences 

By  MOS 

- 

- 

- 

- 

- 

0.20 

0.18 

By  Rater 

Type 

0.31 

0.26  0.25 

0.25 

0.40 

0.28 

0.33 

0.34 

Note.  Cross-MOS  work  activity  ratings  were  not  collected.  Accordingly,  information  on  the  correlations  and  mean 
differences  in  work  activity  ratings  could  not  be  computed. 


MOS  Differentiation  among  Worker-Oriented  Descriptors 

Table  29  presents  the  similarity  and  mean  differences  in  worker-related  characteristics 
ratings  among  the  MOS.  The  correlations  between  the  MOS  for  all  worker-related  characteristics 
were  generally  smaller  than  the  within  MOS  ratings,  except  in  the  case  of  work  values.  For  work 
values,  the  sample-weighted  mean  correlation  was  .79  while  the  average  within  MOS  correlation 
was  only  .70.  In  contrast,  but  consistent  with  the  correlations,  abilities  and  interests  produced 
larger  differences  between  MOS  than  within  MOS  (see  Table  30). 

Summary  of  MOS  Differentiation  Analyses 

The  results  of  the  differentiation  analyses  showed  that  the  descriptors  hold  promise  for 
differentiating  across  MOS.  For  the  work-oriented  descriptors  (i.e.,  work  context),  the  profile 
similarities  across  MOS  were  lower,  on  average,  than  the  similarities  within  MOS.  Among  the 
worker-oriented  descriptors,  this  same  pattern  held  for  work  abilities  and  work  interests.  For 
work  styles  and  values,  however,  the  profile  similarities  across  MOS  were  comparable  to  or 
somewhat  lower  than  the  profile  similarities  within  an  MOS. 
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Table  29.  Profile  Similarity  and  Mean  Differences  in  Worker-Related  Characteristics  Ratings 
among  MOS  by  Domain 


1  IB 

19K 

25U 

31B 

63B 

88M 

Work  Abilities 

1  IB 

.  73/35 

.31 

.62 

.33 

.38 

.41 

19K 

.75 

.51/48 

.46 

.22 

.20 

.19 

25U 

-.06 

.21 

.95/20 

.37 

.32 

.35 

31B 

.46 

.66 

.70 

.86/25 

.27 

.26 

63B 

.40 

.59 

.62 

.58 

.  70/34 

.16 

88M 

.50 

.86 

.49 

.82 

.64 

.79/19 

Work  Styles 

1  IB 

.76/62 

.21 

.27 

.28 

.34 

.26 

19K 

.88 

.67/68 

.20 

.17 

.20 

.14 

25U 

.53 

.64 

.81/39 

.20 

.18 

.23 

31B 

.54 

.65 

.54 

.53/33 

.24 

.20 

63B 

.63 

.73 

.79 

.43 

.23/51 

.21 

88M 

.72 

.82 

.49 

.63 

.65 

.68/17 

Work  Values 

1  IB 

.76/55 

.12 

.21 

.23 

.18 

.15 

19K 

.93 

.66/29 

.23 

.23 

.18 

.17 

25U 

.75 

.77 

.68/22 

.18 

.14 

.16 

31B 

.70 

.79 

.73 

.69/74 

.21 

.19 

63B 

.83 

.84 

.83 

.62 

.78/20 

.12 

88M 

.94 

.92 

.72 

.72 

.82 

.61/23 

Work  Interests 

1  IB 

.86/30 

.16 

.55 

.40 

.32 

.20 

19K 

.95 

.86/34 

.46 

.38 

.24 

.16 

25U 

.35 

.48 

.93/15 

.46 

.36 

.39 

31B 

.68 

.66 

.34 

.857.27 

.49 

.39 

63B 

.75 

.86 

.68 

.42 

.88/20 

.15 

88M 

.89 

.93 

.48 

.61 

.90 

.807.24 

Table  30.  Average  Profile  Similarity  and  Mean  Difference  Statistics  for  the  Worker-Oriented 
Descriptors 


Work  Abilities 

Work  Styles 

Work  Values 

Work  Interests 

Correlations 

By  MOS 

0.55 

0.64 

0.79 

0.67 

By  Rater  Type 

0.76 

0.61 

0.70 

0.86 

Mean  Differences 

By  MOS 

0.32 

0.22 

0.18 

0.34 

By  Rater  Type 

0.30 

0.45 

0.37 

0.25 

The  results  were  less  supportive  on  this  point  when  examining  cross-MOS  mean 
differences  in  ratings.  In  part,  these  results  may  be  a  reflection  of  the  content  of  the 
questionnaires.  Each  questionnaire  may  contain  a  number  of  items  that  were  not  particularly 
useful  for  differentiating  across  MOS,  or  this  particular  sample  of  MOS.  Those  items  would  in 
turn  lower  the  overall  mean  differences  in  the  ratings  reported  for  any  given  questionnaire. 
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Did  Participants  React  Favorably  to  the  Questionnaires? 

Participants  were  asked  to  rate  each  descriptor  domain  on  the  following:  ease  of  use, 
instructions,  clarity  of  statements,  relevance  of  questions  to  MOS,  and  usefulness  for  describing 
MOS  requirements.  Results  from  the  reaction  questionnaires  can  be  found  in  Tables  31  and  32. 
These  tables  include:  (a)  the  mean  response  for  each  MOS  per  item,  (b)  the  SD  for  each  MOS  per 
item,  and  (c)  the  percent  of  responses  that  rated  each  item  a  4  or  5  (defined  here  as  “good”).  We 
also  calculated  the  sample-weighted  mean  and  standard  deviations  for  each  domain  across  all 
MOS  in  order  to  identify  any  MOS  that  exhibited  deviations  from  the  general  pattern  of 
responses  to  the  reaction  items. 

The  work  activities  items  were  generally  viewed  as  easy  to  use,  clear,  relevant,  and 
useful.  This  is  a  positive  sign  for  the  work  activities  items,  as  these  items  were  written 
specifically  for  each  MOS.  The  work  context  ratings  were  generally  high  as  well.  Within  the 
work  context  domain,  the  lowest  ratings  were  typically  assigned  to  the  question,  “Usefulness  for 
describing  the  requirements  of  your  MOS”  (activities  M  =  4.05;  context  M=  3.72).  The  mean  for 
the  work  context  item  regarding  usefulness  is  the  lowest  sample-weighted  mean  for  any  domain 
on  this  item.  It  is  also  the  lowest  sample-weighted  mean  for  any  of  the  reaction  questions. 
Similarly,  the  sample-weighted  mean  for  the  question,  “Relevance  of  questions  to  your  MOS”  in 
the  work  context  domain  was  also  smaller  than  for  most  other  domains.  Therefore,  it  appears  that 
there  was  less  satisfaction  overall  with  respect  to  the  items  comprising  the  work  context 
domains;  however,  these  means  are  still  high  (i.e.,  each  mean  for  the  work  context  reaction 
questions  is  over  3.50). 

Within  the  worker-oriented  domains  (work  abilities,  styles,  interests,  and  values),  the 
ratings  of  work  styles  were  generally  the  highest.  However,  across  all  four  domains,  the  reaction 
ratings  were  typically  high  (i.e.,  greater  than  4.00).  The  sample-weighted  means  for  the  reactions 
to  the  interest  items  were  slightly  lower  than  the  other  domains.  For  work  abilities  and  styles, 

63B  generally  reported  lower  means  than  the  other  MOS.  This  was  not  the  case  for  work 
interests  and  values. 

Reactions  to  work  abilities,  styles,  values,  and  interests  were  generally  more  similar 
across  MOS  than  were  reactions  to  the  work  activities  and  context  reactions  as  indicated  by  the 
lower  SDs  for  worker-oriented  descriptors.  In  contrast,  the  percentage  of  respondents  rating  the 
work-oriented  domain  items  as  “good”  was  generally  higher  than  the  percentage  of  respondents 
rating  the  worker-oriented  domain  items  as  “good.” 

Overall,  participants  rated  the  questionnaires  favorably,  irrespective  of  the  domain,  their 
MOS,  or  their  rank.  Mean  responses  to  all  reaction  measures  items  were  generally  positive,  and 
the  majority  of  all  participants  within  any  MOS  completing  the  questionnaire  rated  the  items  as 
“good.”  In  comparative  terms,  the  worker-oriented  questionnaires  received  less  favorable  ratings, 
on  average,  than  the  work-oriented  questionnaires.  Similarly,  the  worker-oriented  questionnaires 
had  a  lower  percentage  of  respondents  rating  the  questionnaires  on  the  different  reaction 
dimensions  as  “good.”  However,  participant  reactions  were  still  generally  positive  in  absolute 
terms,  with  a  majority  of  the  participants  rating  each  of  the  questionnaires  as  “good.” 
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Table  31.  Respondent  Reactions  to  Work-Oriented  Questionnaires 


Ease  of  Use  (1-5) 

Instructions  (1-5) 

Clarity  (1-5) 

Relevance  to  MOS  (1-5) 

Usefulness  (1-5) 

Domain/MOS 

N 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

%  Good 

Work  Activities 

1  IB 

66 

4.09 

.89 

80.6% 

4.03 

.88 

83.6% 

4.06 

0.89 

83.6% 

4.44 

.79 

88.1% 

4.12 

.79 

83.6% 

19K 

47-48 

4.10 

.81 

81.6% 

4.25 

.79 

87.7% 

4.21 

0.81 

83.7% 

4.31 

.90 

79.6% 

4.06 

1.01 

75.5% 

25U 

158-161 

4.04 

.81 

82.8% 

4.11 

.76 

85.3% 

3.99 

0.86 

78.5% 

4.29 

.86 

82.2% 

4.02 

1.01 

76.7% 

3  IB 

106-108 

4.16 

.81 

80.9% 

4.08 

.86 

80.9% 

4.03 

0.88 

77.3% 

4.10 

.90 

80.0% 

3.89 

.98 

72.7% 

63B 

62-63 

4.03 

.90 

80.0% 

3.98 

.80 

78.5% 

3.98 

0.85 

81.5% 

4.23 

.80 

80.0% 

3.98 

.79 

76.9% 

88M 

93-95 

4.23 

.78 

86.6% 

4.19 

.77 

82.5% 

4.17 

0.88 

80.4% 

4.38 

.84 

84.5% 

4.25 

.87 

85.3% 

Total  M 

4.11 

.83 

4.11 

.81 

4.06 

0.86 

4.28 

.85 

4.05 

.91 

Work  Context 

1  IB 

74-75 

4.27 

.66 

77.7% 

4.25 

.70 

77.7% 

4.08 

0.88 

70.6% 

3.80 

.96 

60.0% 

3.75 

1.05 

57.7% 

19K 

85-86 

4.08 

.69 

77.2% 

4.07 

.72 

76.1% 

3.96 

0.82 

70.7% 

3.91 

.79 

76.1% 

3.72 

.78 

67.4% 

25U 

255-257 

4.23 

.67 

83.6% 

4.33 

.64 

86.8% 

4.02 

0.83 

74.3% 

3.90 

.74 

69.6% 

3.65 

.88 

59.6% 

31B 

106-107 

4.19 

.75 

82.1% 

4.23 

.64 

84.6% 

4.06 

0.77 

73.5% 

4.11 

.78 

76.1% 

3.96 

.84 

72.7% 

63B 

72-73 

3.97 

.78 

64.0% 

3.95 

.76 

66.3% 

3.85 

0.83 

59.6% 

3.64 

1.06 

51.7% 

3.60 

.91 

50.6% 

88M 

147-149 

4.02 

.69 

71.0% 

3.97 

.77 

71.0% 

3.94 

0.74 

67.1% 

3.85 

1.01 

65.3% 

3.72 

1.04 

61.9% 

Total  M 

744-748 

4.14 

.71 

77.3% 

4.17 

.71 

78.8% 

3.99 

0.81 

70.2% 

3.89 

.87 

67.4% 

3.72 

.92 

61.6% 

Table  32.  Respondent  Reactions  to  Worker-Oriented  Questionnaires  -  Abilities,  Styles,  Values,  and  Interests 


Ease  of  Use 

Instructions 

Clarity 

Relevance  to  MOS 

Usefulness 

Domain/MOS 

N 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

%  Good 

Work  Abilities 

1  IB 

63 

4.30 

.56 

76.0% 

4.33 

.51 

78.5% 

4.29 

0.73 

76.0% 

4.41 

.59 

76.0% 

4.40 

.58 

76.0% 

19K 

55-56 

4.11 

.53 

67.6% 

4.11 

.53 

67.6% 

4.04 

0.60 

63.5% 

4.05 

.64 

64.9% 

3.98 

.56 

63.5% 

25U 

161-163 

4.14 

.65 

71.0% 

4.20 

.66 

72.0% 

4.12 

0.72 

71.5% 

4.04 

.70 

68.6% 

4.01 

.71 

68.1% 

31B 

102-103 

4.20 

.68 

79.3% 

4.21 

.68 

78.5% 

4.14 

0.76 

76.7% 

4.15 

.80 

75.0% 

4.12 

.77 

75.0% 

63B 

73-76 

3.97 

.91 

76.7% 

4.00 

.85 

74.2% 

3.97 

0.83 

76.7% 

3.93 

.91 

74.4% 

3.92 

.89 

75.6% 

88M 

98 

4.17 

.75 

68.2% 

4.23 

.59 

69.8% 

4.20 

0.59 

69.0% 

4.03 

.87 

63.6% 

3.98 

.95 

63.6% 

Total  M 

554-561 

4.15 

.69 

72.9% 

4.19 

.66 

73.3% 

4.13 

0.72 

72.3% 

4.09 

.77 

70.0% 

4.06 

.78 

69.8% 

Work  Styles 

1  IB 

54-55 

4.35 

.55 

67.1% 

4.37 

.52 

67.1% 

4.37 

0.52 

67.1% 

4.47 

.57 

67.1% 

4.42 

.57 

67.1% 

19K 

48-49 

4.10 

.59 

56.8% 

4.12 

.60 

58.1% 

4.04 

0.61 

55.4% 

4.02 

.66 

55.4% 

3.94 

.56 

54.1% 

25U 

134-136 

4.18 

.69 

59.9% 

4.18 

.69 

59.4% 

4.21 

0.71 

59.4% 

4.17 

.72 

57.5% 

4.16 

.70 

58.9% 

31B 

92-93 

4.31 

.57 

75.9% 

4.30 

.59 

75.0% 

4.33 

0.60 

75.0% 

4.27 

.57 

75.0% 

4.27 

.59 

73.3% 

63B 

65-67 

3.96 

.91 

67.4% 

3.98 

.86 

65.1% 

3.97 

0.85 

66.3% 

3.99 

.84 

67.4% 

3.95 

.83 

66.3% 

88M 

84-85 

4.32 

.56 

62.8% 

4.29 

.55 

62.8% 

4.28 

0.55 

62.8% 

4.30 

.60 

60.5% 

4.26 

.60 

60.5% 

Total  M 

483-486 

4.21 

.67 

64.7% 

4.21 

.66 

64.2% 

4.22 

0.67 

64.1% 

4.21 

.68 

63.2% 

4.18 

.67 

63.1% 

Table  32.  (continued) 


Domain/MOS 

N 

Ease  of  Use  (1-5) 

Instructions  (1-5) 

Clarity  (1-5) 

Relevance  to  MOS  (1-5) 

Usefulness  (1-5) 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

%  Good 

M 

SD 

% 

Good 

Work  Values 

1  IB 

49-50 

4.14 

.78 

58.9% 

4.06 

.71 

56.2% 

4.04 

.73 

57.5% 

4.04 

.64 

54.8% 

4.06 

.65 

56.2% 

19K 

47 

4.32 

.78 

67.2% 

4.38 

.64 

67.2% 

4.26 

.71 

62.7% 

4.23 

.67 

64.2% 

4.15 

.62 

64.2% 

25U 

168-170 

4.17 

.69 

65.7% 

4.13 

.69 

64.8% 

4.06 

.80 

63.1% 

4.04 

.72 

62.7% 

4.03 

.80 

61.4% 

31B 

75 

4.21 

.79 

61.3% 

4.13 

.89 

58.6% 

4.12 

.85 

57.7% 

4.15 

.80 

59.5% 

4.11 

.80 

58.6% 

63B 

43-44 

4.25 

.58 

60.3% 

4.26 

.69 

57.4% 

4.23 

.57 

60.3% 

4.20 

.63 

57.4% 

4.23 

.60 

85.3% 

88M 

102-104 

4.22 

.56 

68.8% 

4.26 

.52 

69.4% 

4.12 

.69 

67.4% 

4.10 

.64 

66.7% 

4.07 

.61 

67.4% 

Total  M 

487-490 

4.21 

.69 

64.3% 

4.18 

.70 

63.2% 

4.11 

.75 

62.1% 

4.10 

.70 

61.6% 

4.10 

.70 

61.5% 

Work  Interests 

1  IB 

56-57 

4.09 

.87 

65.8% 

4.18 

.68 

68.5% 

4.09 

.79 

63.0% 

3.96 

.84 

58.9% 

4.02 

.81 

63.0% 

19K 

52-53 

4.11 

.97 

67.2% 

4.17 

.89 

67.2% 

4.19 

.90 

67.2% 

4.06 

.98 

64.2% 

3.96 

.83 

64.2% 

25U 

183-185 

4.19 

.58 

72.9% 

4.22 

.57 

73.3% 

4.15 

.72 

69.5% 

4.08 

.73 

67.0% 

4.00 

.78 

67.4% 

31B 

93-94 

4.20 

.75 

76.6% 

4.18 

.76 

75.7% 

4.13 

.83 

73.0% 

4.05 

.87 

73.0% 

4.06 

.85 

72.1% 

63B 

52-53 

4.13 

.71 

66.2% 

4.21 

.70 

67.7% 

4.11 

.67 

67.7% 

3.94 

.72 

58.8% 

3.92 

.70 

58.8% 

88M 

116-119 

4.20 

.56 

76.4% 

4.20 

.56 

74.3% 

4.14 

.71 

75.0% 

3.95 

.86 

69.4% 

3.95 

.83 

70.1% 

Total  M 

558-561 

4.17 

.69 

72.2% 

4.20 

.66 

72.2% 

4.14 

.76 

70.0% 

4.02 

.82 

66.5% 

3.99 

.80 

66.9% 

V.  Summary  of  Findings  and  Discussion 
Major  Findings  and  Limitations  from  the  Field  Test 

The  purpose  of  the  field  test  was  to  evaluate  the  prototype  job  analysis  approach.  Our 
evaluation  of  the  approach  focused  on  answering  the  following  questions: 

•  Were  the  jobs  rated  reliably  using  the  field  test  questionnaires? 

•  Did  incumbents  and  supervisors  agree  in  their  ratings? 

•  Did  the  ratings  differentiate  among  jobs? 

•  Did  participants  react  favorably  to  the  questionnaires? 

Overall,  the  results  of  the  field  test  were  encouraging,  as  evidenced  by  the  following: 

The  multi-rater  reliability  estimates  were  consistently  .80  or  greater,  with  many  .90  and 
above,  indicating  that  the  mean  ratings  evidence  sufficient  reliability  to  support  their  use  in 
operational  decision-making  (e.g.,  clustering  MOS).  These  estimates  were  generally  comparable 
to  or  exceeded  those  obtained  on  corresponding  questionnaires  administered  to  Army  supervisors 
during  an  evaluation  of  the  use  of  the  existing  0*NET  system  to  describe  Army  jobs.  In 
particular,  the  estimates  for  worker  abilities  obtained  using  the  prototype  approach  were 
consistently  higher  than  those  observed  using  the  0*NET  work  abilities  questionnaire  (a  mean 
multi-rater  ICC  of  .95  using  the  prototype  approach  versus  .75  for  the  0*NET). 

Overall,  there  was  greater  within  group  variability  than  between  group  variability, 
indicating  that  most  of  the  systematic  variance  in  ratings  was  more  likely  a  function  of  yet  to  be 
examined  factors  (e.g.,  duty  position),  rather  than  rater  type.  Differences  between  incumbent  and 
supervisor  ratings,  on  average,  were  generally  low.  The  absolute  standardized  mean  differences  in 
ratings  between  the  two  rater  types  were,  on  average,  about  one-third  of  a  standard  deviation  or 
smaller  (mean  \d\  <  .33),  compared  to  the  standard  deviations  within  any  rater  type,  which  were 
typically  .70  or  greater.  Further,  there  was  no  evidence  that  the  differences  followed  a  systematic 
pattern  (i.e.,  one  group  consistently  rated  the  descriptors  within  a  given  domain  higher  or  lower  than 
the  other  group).  Similarly,  the  correlations  in  the  ratings  profile  between  the  two  rater  types  were 
generally  high,  on  average  (mean  r  of  .53  or  greater),  with  correlations  in  the  .80s  for  work  activities, 
evidence  that  incumbents  and  supervisors  were  generally  consistent  in  the  relative  importance  they 
assigned  the  job  descriptors  within  any  given  domain. 

Ratings  from  the  questionnaires  demonstrated  promise  for  differentiating  among  MOS, 
although  the  differences  were  small  in  several  instances,  at  least  in  absolute  terms.  With  the 
exception  of  work  styles  and  values,  the  mean  correlations  in  rating  profiles  across  MOS  for  any 
single  domain  were  consistently  lower,  on  average,  than  the  correlations  between  rater  types  within 
an  MOS  (with  differences  in  mean  r  ranging  from  .09  to  .22).  A  similar  pattern  of  results,  however, 
was  not  evident  when  examining  the  average  absolute  standardized  mean  differences,  where  the 
mean  differences  between  MOS  were  generally  comparable  to  or  lower  than  those  reflecting 
differences  between  rater  types  within  an  MOS  (the  differences  in  mean  |*/|  ranged  from  .02  to  .22). 
Among  the  worker-oriented  questionnaires,  the  ones  demonstrating  the  greatest  potential,  on  average, 
for  differentiating  across  MOS  were  worker  abilities  and  interests,  at  least  for  this  particular  sample 
of  MOS  (Abilities:  mean  r  =  .55,  mean  \d\  =  .32;  Interests:  mean  r  =  .67;  mean  \d\  =  .34). 
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Participants  ’  reactions  to  the  questionnaires  and  their  content  were  generally  positive, 
both  in  terms  of  their  ease  of  use  and  their  relevance  for  describing  their  MOS.  About  62%  or 
greater  of  the  participants,  on  average,  found  the  questionnaires  easy  to  use  and  the  instructions 
and  content  clear,  with  upwards  of  87%  of  participants  reacting  that  way  to  the  work  activities 
questionnaires.  Similarly,  62%  or  more  of  the  participants,  on  average,  rated  the  questionnaires 
as  useful  and  relevant  to  describing  their  MOS,  with  upwards  of  88%  of  participants  rating  the 
work  activities  questionnaires  in  that  manner.  Overall,  participants  reacted  more  favorably  to  the 
work-oriented  questionnaires,  in  particular  the  work  activities  questionnaire,  than  the  worker- 
oriented  questionnaires.  This  finding  is  not  surprising,  given  that  the  work-oriented 
questionnaires,  specifically  those  measuring  work  activities,  contained  the  most  job-specific 
content. 

Although  the  results  of  the  field  test  were  generally  promising,  this  research  was  not 
without  its  limitations.  Specifically: 

The  response  rates  to  the  questionnaires  were  generally  low,  particularly  for  incumbents . 
Among  incumbents,  response  rates  were  typically  5%  or  less.  The  response  rates  among 
supervisors  tended  to  be  two  to  three  times  that  rate  (10%  to  15%),  with  a  high  close  to  31%. 
There  were  also  notable  differences  in  response  rates  by  MOS,  with  25U  generally  evidencing 
the  highest  response  rates  and  1  IB  and  19K  demonstrating  the  lowest  response  rates.  Although 
low,  these  response  rates  were  consistent  with  those  typically  encountered  by  the  Anny’s 
Occupational  Analysis  Program  (OAP). 

The  ratings  collected  did  not  permit  an  examination  of  the  similarities  and  differences  in 
work  activities  by  MOS.  Although  ratings  were  collected  on  work  activities  during  the  field  test, 
these  data  were  on  the  lower-order  (or  more  job-specific)  activities.  As  a  result,  no  data  were 
available  to  evaluate  the  psychometric  properties  and  discriminate  validity  associated  with  a 
questionnaire  measuring  higher-order  (or  cross  job)  work  activity  descriptors.  A  preliminary  set 
of  those  descriptors,  based  on  the  Job  Activities  and  Task  Categories  generated  for  Project 
SYNVAL,  were  developed  for  this  project  and  used  during  the  0*NET  evaluation  (see  Russell  et 
al.,  2008).  Because  priorities  had  to  be  made,  a  questionnaire  measuring  those  cross  job  work 
activity  descriptors  was  not  administered  in  the  field  test.6  However,  some  evidence  for  the 
potential  value  of  such  a  measure,  albeit  indirect,  was  evident  in  the  results  for  the  work  interests 
questionnaire.  The  work  interests  questionnaire  measured  the  importance  of  interests  in 
performing  various  higher-order  work  activities  required  for  the  job.  The  content  of  the 
questionnaire  was  based  on  a  slightly  modified  version  of  the  0*NET’s  Generalized  Work 
Activities  (GWA)  taxonomy.  In  general,  the  work  interests  measure  evidenced  higher  multi-rater 
reliability  estimates,  greater  discriminant  validity,  and  more  favorable  participant  reactions  than 
the  other  worker-oriented  questionnaires. 


6  Further  evidence  for  the  utility  of  the  cross  job  work  activity  descriptors  can  be  found  in  a  recently  completed 
project  that  used  an  expanded  version  of  these  descriptors  in  a  content  validation  of  the  ASVAB  (Waters,  Russell, 
Shaw,  Allen,  Sellman,  &  Geimer,  2009). 
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Although  the  evidence  for  discriminant  validity  was  promising,  the  differences  in  ratings 
by  MOS  from  any  single  domain  were  generally  small  in  absolute  terms.  For  example,  the 
average  absolute  standardized  mean  differences  by  MOS  ranged  from  .18  (Work  Context  and 
Work  Values)  to  .34  (Work  Interests).  There  are  several  potential  explanations  for  this  finding: 

(a)  the  nature  of  the  MOS  sampled;  (b)  the  use  of  importance  ratings,  as  opposed  to  level  ratings 
or  some  alternative  kind  of  rating  exercise  with  greater  potential  to  differentiate  across  MOS;  (c) 
the  questionnaires  contained  a  number  of  items  not  useful  for  differentiating  across  MOS,  or  at 
least  this  particular  sample  of  MOS;  and  (d)  the  use  of  web-based  surveys  to  collect  ratings,  as 
opposed  to  some  alternative  data  collection  method  (e.g.,  a  workshop). 

Looking  at  the  Army’s  Model  for  Generating  Job  Analysis  Data:  Beyond  the  Descriptors 

The  work  in  this  project  has  focused  primarily  on  developing  and  field  testing  a  job 
analysis  approach  to  describe  Army  jobs  that  could  be  implemented  Anny-wide  to  support  the 
many  needs  of  its  classification  research  program,  from  clustering  MOS  to  criterion 
measurement  development  to  validity  generalization  or  transportation.  Although  personnel 
classification  served  as  the  primary  driver,  many  elements  of  the  approach  were  designed  in  a 
way  that  they  could  potentially  support  the  Army’s  other  job  analysis  needs  as  well,  particularly 
training  development.  An  implicit  assumption  has  been  that  the  Anny’s  current  model  for 
collecting  job  analysis  data  would  stay  the  same.  Key  elements  of  this  model  are  that  (a)  MOS 
proponents  are  responsible  for  creating  the  job  analysis  descriptors,  using  guidance  provided  in 
Army  regulations,  and  (b)  the  primary  data  collection  strategy  is  through  web-based 
questionnaires.  In  this  section,  we  discuss  why  it  is  important  to  consider  alternative  models  for 
conducting  occupational  analysis  in  the  Army  and  to  discuss  how  they  might  be  changed  to  make 
it  more  feasible  to  maintain  a  new  Army-wide  job  analysis  system.  Some  of  these  strategies  have 
been  incorporated  into  the  prototype  questionnaire  (e.g.,  use  of  standardized,  cross-job 
descriptors). 

Barriers  to  Collecting  Army  Job  Analysis  Information 

Historically,  it  has  been  difficult  for  the  Army  to  expend  the  personnel  resources  required 
to  update  job  analysis  information.  This  includes  the  time  required  for  Army  personnel  to  update 
the  infonnation  that  would  go  into  a  questionnaire  as  well  as  the  time  it  takes  for  respondents  to 
complete  it.  Over  the  past  decade  as  the  Anny’s  operational  tempo  has  increased  dramatically, 
this  constraint  has  become  particularly  severe,  resulting  in  outdated  job  analysis  infonnation  for 
many  MOS.  No  matter  how  ideal  the  descriptors  are,  an  Army-wide  job  analysis  system  that 
does  not  effectively  minimize  practical  baniers  to  the  generation  and  collection  of  this 
information  will  not  be  effective. 
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Strategies  to  Minimize  Burden  on  Army  Personnel  for  Developing  Job  Descriptors 

Use  Standardized  Job  Descriptors 

Proponents  under  the  Army  Training  and  Doctrine  Command  (TRADOC)  are  responsible 
for  maintaining  job  analysis  information  for  their  respective  MOS.  When  requested  by  the 
proponent,  ARI  provides  technical  and  administrative  support  through  its  OAP.  The  OAP 
provides  assistance  in  developing  descriptor  content,  programming  and  administering  web-based 
questionnaires  using  the  InterFonn  software  application,  and  data  analysis.  An  advantage  of  the 
job  descriptors  developed  in  the  present  research  is  that  most  of  them,  with  the  exception  of  the 
lower-order  (or  more  job-specific)  work  activities,  are  standardized  across  MOS.  This  feature  (a) 
reduces  the  burden  on  individual  proponents  to  develop  job-specific  information,  and  (b)  enables 
the  Army  to  examine  the  similarities  and  differences  across  MOS,  by  providing  a  common  basis 
of  comparison.  The  results  of  the  field  test,  and  those  of  the  Army’s  0*NET  evaluation  project, 
demonstrate  that  using  a  standardized  approach  for  measuring  targeted  descriptor  domains  yields 
sufficiently  reliable  and  valid  data  for  characterizing  the  similarities  and  differences  in  MOS, 
supplementing  the  kinds  of  more  job-specific  infonnation  collected  under  the  existing  system, 
with  minimal  additional  development  costs  to  the  Army. 

Avoid  Unnecessarily  Detailed  Job-Specific  Descriptors 

The  lower-order  work  activities  identified  in  the  present  research  were  intended  to  be  less 
detailed,  or  specific,  than  the  tasks  prescribed  in  Army  Regulation  350-70  and  TRADOC 
Pamphlet  350-70-6.  Defining  lower-order  work  activities  in  a  broader  manner  makes  the 
activities  relevant  to  many  or  all  MOS  rather  than  only  a  small  number.  Taking  this  approach  is 
advantageous  for  several  reasons.  First,  the  prototype  approach  produces  descriptors  detailed 
enough  to  support  criterion  measurement  development,  as  well  as  training  development,  but  that 
can  retain  their  relevancy  longer  than  more  traditional,  detailed  task  statements.  Second,  it  results 
in  a  smaller  number  of  descriptors,  making  them  easier  to  review,  maintain,  and  collect  data  on, 
compared  to  the  Army’s  traditional  approach.  Third,  specifying  work  activity  descriptors  at  a 
slightly  more  general  level  is  consistent  with  the  emphasis  that  several  Army  proponents  (e.g., 
medics)  have  placed  on  principles-based  training,  rather  than  training  based  on  a  highly  detailed 
series  of  tasks  and  proceduralized  steps  for  accomplishing  them.  The  shift  in  training  program 
orientation  is  consistent  with  TRADOC ’s  increasing  emphasis  on  outcomes-based  training 

Although  the  prototype  approach  provides  a  useful  model  for  generating  lower-order 
work  activities,  input  from  Army  job  incumbents  or  supervisors  will  be  needed  to  ensure  that  any 
work  activity  statements  collected  for  future  job  analysis  efforts  are  current  and  reasonably 
comprehensive.  In  the  present  research,  we  sought  less  input  from  job  incumbents  or  supervisors 
than  would  most  likely  be  required  for  an  operational  program  because  previous  research  on  the 
MOS  sampled  in  the  present  research  provided  great  insight  into  the  characteristics  of  those 
MOS.  The  currency  of  input  from  retired  SMEs  cannot  be  assured.  Any  new  system  would 
thereby  benefit,  and  would  be  more  cost  effective  in  the  long  run,  were  existing  Army 
Regulations  revised  to  produce  content  that  best  meets  its  needs  and  constraints.  As  with  any 
new  system,  initial  efforts  will  demand  greater  labor  resources  than  subsequent  efforts,  once 
inefficiencies  have  been  identified  and  revised. 


57 


We  estimate  that  the  cost  for  generating  the  lower-order  work  activity  descriptors 
following  the  prototype  approach  would  initially  run  about  $15-25K  per  MOS  (in  FY  2010 
dollars).  This  cost  estimate  assumes  existing  expertise  with  the  prototype  approach  and  modest 
support  from  SMEs  with  knowledge  of  the  kinds  of  work  activities  performed  by  entry-level 
Soldiers  in  the  MOS.  Over  time,  the  cost  per  MOS  would  decrease  as  the  pool  of  work  activity 
descriptors  expand  to  reflect  the  kinds  of  activities  required  of  a  representative  sample  of  MOS. 
At  that  point,  existing  work  activity  descriptors  could  be  re-used  or  modified,  as  appropriate 
(e.g.,  by  simply  changing  the  kind  of  weapon-equipment-tool  used),  to  describe  similar  MOS  at 
minimal  cost  to  the  Army.  Please  note  that  the  aforementioned  cost  estimate  does  not  take  into 
account  methods  of  collecting,  maintaining,  and  analyzing  data. 

Strategies  to  Increase  Effectiveness  of  Data  Collection  Approach 

A  primary  feature  of  the  Army’s  existing  job  analysis  model  is  that  information  is 
collected  from  incumbents  and  supervisors  via  web-based  questionnaires.  According  to  Army 
Regulation  370-50,  updated  job  analysis  information  should  be  collected  every  3  years.  This 
guidance  has  largely  been  difficult  to  follow  for  the  reasons  cited  above.  Even  for  those  MOS 
with  updated  information.  Army  job  analysis  surveys  typically  yield  a  response  rate  of  just  10%- 
12%.  Our  experience  in  the  field  test  was  similar. 

Two  major  obstacles  to  obtaining  larger  response  rates  are  (a)  reaching  Soldiers  and  (b) 
motivating  them  to  respond.  In  the  present  research,  we  e-mailed  links  to  the  questionnaires  via 
Soldier  AKO  email  accounts.  This  process  is  similar  to  the  current  operational  system  and 
resulted  in  comparably  low  response  rates.  As  we  have  found  in  other  web-based  data  collections 
with  Soldiers,  it  appears  that  many  Soldiers  do  not  routinely  access  their  AKO  accounts.  We  do 
not  have  a  good  solution  for  an  alternative  approach.  One  would  hope,  however,  that  as  the 
Army  increasingly  relies  upon  AKO  to  communicate  with  Soldiers,  Soldiers  will  use  their  AKO 
accounts  more  frequently  and  thus  be  more  reachable  through  this  method. 

Regarding  efforts  to  enhance  Soldiers’  motivation  to  respond  to  job  analytic  surveys, 
because  the  Army  does  not  allow  certain  forms  of  incentives  (e.g.,  drawings  for  prizes  or  gift 
cards),  there  are  fewer  tactics  available  in  the  Army  than  in  other  contexts  (e.g.,  private-sector 
organizations  and  the  general  public).  Researchers  typically  hope  that  emphasizing  the 
importance  of  a  project’s  objectives  will  increase  Soldier  participation.  Along  with  this 
approach,  the  present  project  attempted  to  make  Soldiers  more  likely  to  participate  by 
minimizing  the  time  demands  associated  with  completing  its  surveys.  We  employed  a  matrix 
sampling  approach,  whereby  not  all  questionnaires  are  administered  to  all  participants,  to  limit 
the  time  required  of  any  given  Soldier.  Even  though  the  resulting  groups  of  questionnaires 
required  20  minutes  or  less  to  complete,  our  response  rates  were  still  generally  low.  Though  we 
continue  to  favor  strategies  to  limit  questionnaire  length  or  administration  time  (e.g.,  reducing 
the  number  of  scales  administered,  using  a  matrix  sampling  approach),  response  rates  of  the 
present  project  suggest  that  such  strategies  may  not  significantly  increase  response  rates. 
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Consider  Collecting  Data  in  Workshops 

An  alternative  to  the  questionnaire-based  approach  used  in  the  field  test  and  by  OAP 
would  be  to  collect  some  or  all  of  the  required  ratings  through  one  or  more  workshops  with 
participating  Army  job  incumbents  or  supervisors.  The  present  research  found  promising,  but 
small  cross-MOS  differences.  There  are  several  possible  reasons  for  this,  to  include  the  MOS 
sampled  in  the  research.  Another  possibility  is  that  a  questionnaire-based  approach  is  not 
conducive  to  reflecting  the  nuances  in  a  5 -point  rating  scale  that  are  essential  to  capturing  cross- 
MOS  differences;  this  was  the  primary  motivation  for  including  a  ranking  exercise  in  the 
questionnaires,  first  piloted  and  then  dropped  prior  to  the  field  test.  It  is  possible  that  judgments 
made  in  a  workshop  setting  would  result  in  greater  discrimination,  because  of  the  increased  time 
for  training  and  discussion  among  workshop  participants.  Such  workshops  were  used 
successfully  in  the  Army  0*NET  evaluation  and  could  hold  promise  as  an  alternative  to  a  large- 
scale  questionnaire-based  approach. 

There  are  pros  and  cons  to  any  given  data  collection  method,  including  workshops. 
Whereas  workshops  could  result  in  more  reliable  and  discriminant  ratings  than  a  questionnaire- 
based  approach,  they  would  require  more  time  from  a  smaller  and  potentially  less  representative 
sample  of  SMEs.  Any  method  selected  will  result  in  trade-offs  between  costs  and  the  quality  of 
the  infonnation  obtained,  however  measured. 

Collect  SL1  Occupational  Analysis  Data  Only  From  Supervisors 

In  this  field  test  as  in  other  questionnaire-based  research  involving  Soldiers,  response 
rates  were  consistently  higher  among  senior  than  junior  Soldiers.  Although  we  observed 
differences  in  the  ratings  made  between  incumbents  and  supervisors,  we  found  minimal  evidence 
that  the  differences  were  systematic  or  attributable  to  substantive  position  differences  and  not 
other  factors.  If  incumbents  are  not  required  to  collect  accurate  job  information,  then  they  could 
be  excluded  from  data  collection,  assuming  the  absence  of  reasons  to  include  their  input  (e.g.,  if 
ratings  are  expected  to  differ  by  rating  source).  As  with  the  other  suggestions  for  data  collection, 
decisions  regarding  sampling  strategy  should  consider  balancing  anticipated  information  quality 
and  information  gathering  efficiency  according  to  project  objectives. 

Ensuring  Training  Requirements  are  Addressed 

Although  the  primary  objective  of  the  present  research  was  to  ensure  that  the  Army 
collects  job  analysis  data  that  supports  classification  research,  it  is  important  not  to  lose  sight  of 
the  importance  of  these  data  for  training  systems  development.  There  are  well-established  Anny 
regulations  and  guidelines  associated  with  the  Army’s  current  job  analysis  system  that  are  driven 
by  training  needs.  These  are  engrained  in  the  current  system  even  if  they  are  not  strictly 
followed.  For  example,  some  proponents  have  moved  toward  training  strategies  that  are  no 
longer  strictly  task-focused.  An  example  is  medics  who  are  increasingly  being  trained  on  medical 
principles  that  allow  them  to  problem-solve  when  proceduralized  task  knowledge  is  insufficient. 

Now  may  be  the  right  time  to  involve  TRADOC  proponents  in  evaluating  potential 
changes  to  the  existing  system.  Do  the  processes  being  developed  as  part  of  ARI’s  line  of 
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research  adequately  support  the  Army’s  current  training  requirements?  Indeed,  one  can  also  ask 
if  the  Army’s  current  job  analysis  process  satisfy  these  requirements.  To  the  extent  to  which  they 
do  not,  what  modifications,  either  small  or  large,  should  be  made?  It  is  certainly  conceivable  that 
the  system  field  tested  in  the  present  research  effort  would  provide  some  but  not  all  the 
information  needed  to  support  TRADOC’s  needs.  In  that  case,  how  would  this  new  system  feed 
into  the  existing  processes  specified  by  Army  regulations  and  TRADOC  pamphlets?  Addressing 
such  questions  would  clearly  be  key  to  any  implementation  plan. 


VI.  Recommendations  for  Additional  Research 

As  illustrated  in  Figure  1  (pg.  2  of  this  report),  the  next  step  in  the  ACRP’s  roadmap  calls 
for  using  the  prototype  job  analysis  approach  to  collect  data  on  a  larger  sample  of  MOS.  As  part 
of  such  an  effort,  we  propose  three  areas  for  further  research. 

•  Refine  the  job  analysis  descriptors  and  rating  scales 

•  Evaluate  alternative  data  collection  strategies 

•  Collect  data  on  cross-job  descriptors  from  a  larger  sample  of  MOS  to  enable  MOS 
clustering 

As  discussed  in  the  remainder  of  this  report,  we  believe  that  the  prototype  job  analysis  approach 
used  in  the  field  test  can  be  refined  to  maximize  its  effectiveness  and  efficiency.  An  assessment 
of  its  utility  for  classification  can  be  cost-effectively  accomplished  through  collection  of  data 
from  a  relatively  large  sample  of  MOS  using  descriptor  domains  that  are  invariant  across  MOS. 

Any  significant  variations  on  the  current  prototype  process  (e.g.,  adding  a  different  type 
of  rating  scale  or  collecting  data  in  workshops  versus  web-based  questionnaires)  should  be  tested 
before  their  incorporation  into  the  process.  If  a  revision  to  the  system  affects  work-oriented 
descriptor  domains,  it  could  be  initially  evaluated  among  a  subsample  of  the  MOS.  Revisions 
that  pertain  to  worker-oriented  descriptor  domains  that  would  be  part  of  an  MOS  clustering  data 
collection  should  be  evaluated  using  an  experimental  design. 

Refine  the  Descriptors  and  Rating  Scales 


Cross-Job  Descriptors 

It  is  likely  that  some  of  the  worker-oriented  domains  include  descriptors  that  do  not 
differentiate  well  among  MOS.  Identifying  and  eliminating  such  descriptors  from  the  job 
analysis  process  would  improve  the  overall  utility  of  the  overall  system  for  classification.  It 
would  not  be  wise  to  discard  problematic  descriptors  based  on  the  small  sample  of  MOS  used  in 
the  present  field  test,  but  the  removal  of  descriptors  that  fail  to  distinguish  among  a  larger 
number  of  MOS  should  be  considered.  Discarding  such  descriptors  may  not  only  improve  the 
differential  validity  of  the  overall  set  of  descriptors  but  may  also  enhance  Soldier  reactions  to  the 
job  analysis  process  by  removing  statements  that  are  either  irrelevant  or  overly  common  across  a 
wide  range  of  MOS. 
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Work  Activities 


Our  initial  efforts  to  develop  occupation-specific  knowledge  and  skills  descriptors  were 
discontinued  because  they  tended  to  overlap  with  work  activity  descriptors.  A  more  useful 
approach  for  incorporating  this  kind  of  information  would  be  to  combine  the  two  domains  in 
new,  useful  ways,  per  one  of  the  earlier  recommendations  of  the  ACRP.  A  similar 
recommendation  could  be  made  with  regards  to  weapons-equipment-tools-technology.  Table  33 
contains  illustrative  examples  of  descriptors  that  integrate  these  two  domains  into  work  activity 
descriptors,  based  on  the  0*NET’s  Detailed  Work  Activities  (DWAs)  (Dietrich  et  ah,  2002;  The 
National  Center  for  0*NET  Development,  2003). 

Table  33.  Examples  of  Work  Activity  Descriptors  that  Incorporate  Knowledge  and  Tools- 
Based  Descriptors 

Domain _ Examples _ 

Occupation-Specific  Knowledges  •  Understand  detailed  electronic  design  specifications 

and  Skills  •  Understand  technical  information  for  electronic  repair  work 

_ •  Understand  technical  operating,  service  or  repair  manuals _ 

Weapons-Equipment-Tools-  •  Operate  hoist,  winch,  or  hydraulic  boom 

Technology’  •  Set  up  and  operate  variety  of  machine  tools 

•  Use  electrical  or  electronic  test  devices  or  equipment 

•  Use  hand  or  power  tools 

•  Use  lifting  equipment  in  vehicle  repair  setting 


Other  examples  can  be  found  in  the  lower-order  work  activity  statements  developed  for 
the  present  field  test.  Taking  this  approach  would  make  generating  the  job-specific  content 
(lower-order  work  activities)  needed  to  support  the  Army’s  classification  research  program 
easier.  Once  a  standardized  series  of  templates  of  work  activity  statements  for  describing  a 
targeted  subset  of  jobs  (e.g.,  mechanical  or  electronics  repair)  has  been  developed,  one  could 
then  easily  produce  job-specific  statements  by  inserting  content  specific  to  a  job.  The  kinds  of 
weapons/equipment/tools/technology  used  or  occupation-specific  knowledge  required  of  a  job 
typically  differentiates  jobs  at  the  lower  levels  of  detail  to  a  greater  extent  than  what  a  Soldier 
does  with  those  items  (e.g.,  maintaining,  trouble-shooting,  repairing,  operating). 

Ratings 

It  can  be  argued  that  the  conceptual  distinction  between  ratings  of  a  task’s  job  importance 
and  consequences-for-mission  is  subtle.  In  the  present  research,  importance  and  mission 
criticality  ratings  correlated  at  .97  among  supervisors  and  at  .92  among  incumbents.  Clearly,  it  is 
not  necessary  to  collect  both  sets  of  ratings.  As  discussed  previously,  reducing  survey  length  is 
desirable  and  can  be  achieved  by  reducing  the  number  of  rating  scales. 

If  only  the  importance  rating  scale  is  retained,  it  could  be  revised  to  promote  greater 
differentiation  among  jobs.  For  example,  supervisors  could  be  asked  to  rate  worker-related 
descriptors  in  comparison  to  other  MOS  (e.g.,  How  important  is  this  work  style/interest/value  to 
the  job  of  first  tenn  Soldiers  in  this  MOS  relative  to  other  MOS?).  We  would  not  advise  using 
such  a  scale  with  first-term  incumbents,  but  NCOs  would  be  in  a  better  position  to  make  such 
judgments. 
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Further  consideration  of  the  difficulty-to-learn  scale  is  also  warranted  prior  to  the 
collection  of  additional  data.  Because  the  scale  was  developed  without  input  from  Anny 
personnel  responsible  for  the  design  of  training  programs,  it  may  have  questionable  relevance  for 
future  TRADOC  initiatives.  Before  the  scale  is  used  in  future  research,  input  should  be  obtained 
from  TRADOC  personnel  on  issues  related  to  the  scale,  such  as  whether  focusing  on  classroom 
training  limits  the  generalizability  of  results  found  from  using  the  scale. 

Investigate  Alternative  Data  Collection  Strategies 

At  least  for  SL1  job  analysis  information,  the  field  test  results  suggest  that  it  may  not  be 
necessary  to  collect  both  incumbent  and  supervisor  data.  We  suggest  using  the  proposed  MOS 
clustering  data  collection  to  confirm  (or  not)  these  results.  Whether  collecting  ratings  via 
questionnaires  or  in  workshop  settings,  if  SL1  Soldiers  are  not  required  in  collecting  accurate 
information,  they  should  be  excluded  from  the  process. 

Another  strategy  concerns  collecting  various  types  of  descriptor  information.  The 
workshop  method  could  be  implemented  in  various  ways,  including  a  single  workshop  with  a 
sufficient  number  and  sampling  of  SMEs  to  represent  variations  in  the  MOS.  Alternatively,  a 
series  of  workshops  could  be  held  at  multiple  locations  to  get  closer  to  concentrations  of  SMEs 
in  the  MOS.  The  research  would  be  most  informative  if  multiple  strategies  could  be  used  for  the 
same  MOS  so  the  results  would  be  directly  comparable.  The  comparison  could  examine  quality 
of  information  as  well  as  cost  and  SME/proponent  reactions. 

Finally,  future  research  may  want  to  assess  the  extent  to  which  the  utility  of  different 
methods  of  data  collection  depends  upon  the  type  of  data  collected.  For  example,  it  could  be  that 
data  on  work-oriented  descriptors  are  most  efficiently  collected  using  questionnaires  but  that 
worker-oriented  data  suitable  for  MOS  clustering  work  is  more  effectively  collected  in 
workshops. 


Collect  Data  on  a  Larger  Sample  of  MOS  and  Use  to  Cluster  MOS 

In  the  near-term,  collecting  job  analysis  ratings  on  a  larger  sample  of  MOS  would 
potentially  serve  two  purposes.  The  first  purpose  would  be  to  cluster  MOS  in  order  to  select 
which  to  target  in  criterion-related  validation  research.  The  second  purpose  would  be  to  identify 
or  develop  end-of-training  criteria  that  sufficiently  support  the  Army’s  personnel  classification 
research  but  require  fewer  resources  to  construct  and  administer  [e.g.,  the  occupational  judgment 
tests  developed  by  Legree,  Psotka,  Bludau,  and  Gray  (2009)].  In  the  long-term,  collecting  job 
analytic  ratings  on  a  large  sample  of  MOS  would  allow  the  Army  to  evaluate  procedures  for 
generalizing  or  transporting  criterion-related  validation  evidence  obtained  from  a  smaller  sample 
of  MOS  to  a  larger  one.  This  would  be  useful  for  the  development  or  updating  of  prediction 
equations  to  be  used  in  classifying  new  Soldiers,  for  example. 

For  the  purposes  of  meeting  the  near-term  objective  of  clustering  MOS,  an  expanded 
sample  of  MOS  would  need  to  be  larger  than  the  number  of  focal  MOS  to  be  included  in  the 
proposed  validation  research.  The  ACRP  recommended  that  the  sample  of  focal  MOS  for  the 
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criterion-related  validation  research  would  comprise  20-30  jobs.  Accordingly,  job  analysis  data 
would  need  to  be  collected  from  an  additional  45-55  MOS  to  ensure  that  a  sample  will  be  large 
enough  to  inform  clustering  or  validity  generalization  analyses.  Below  is  a  proposed  procedure 
for  collecting  data  on  a  large  sample  of  MOS. 

Activity  1:  Select  the  MOS  to  be  sampled. 

This  would  first  involve  an  identification  of  relevant  factors  upon  which  to  base  decisions 
about  the  inclusion  or  exclusion  of  certain  MOS  from  sampling  plans.  Such  factors  would  reflect 
the  demands  and  resources  associated  with  a  research  project  (i.e.,  scientific  or  technical,  Army 
policy  and  personnel  management  objectives,  resource  constraints).  Once  a  set  of  factors  had 
been  chosen,  MOS  would  then  be  rated  (or  ranked)  according  to  the  factors  and  a  final  set  of  45- 
55  MOS  would  be  selected.  Any  infonnation  on  existing  or  proposed  MOS  clusters  could  be 
useful  in  this  selection  process. 

Activity  2:  Select  the  descriptor  domains  to  be  measured. 

Ideally,  ratings  would  be  collected  on  all  the  descriptor  domains.  However,  priorities  will 
likely  need  to  be  made  to  minimize  the  data  collection  burden  placed  on  Army  personnel. 
Accordingly,  we  would  recommend  prioritizing  the  domains  as  follows  (from  most  to  least 
important): 


•  Higher-order  (or  cross  job)  work  activities 

•  Work  interests 

•  Work  styles  and  values 

•  Work  abilities 

This  prioritization  is  based  on  the  field  test  results,  as  well  as  earlier  recommendations  of 
the  ACRP,  regarding  which  domains  represent  the  “best  bets”  for  clustering  MOS  in  ways  that 
would  maximize  their  utility  for  personnel  classification  research.  Collecting  ratings  on  the  first 
three  domains  would  be  particularly  useful  because  those  data  could  then  be  used  to  develop  a 
preliminary  set  of  rationally-weighted  composites  for  classifying  new  Soldiers  to  MOS,  based  on 
scores  from  new  and  emerging  predictor  measures,  such  as  the  Work  Preferences  Assessment 
(WPA)  or  the  Tailored  Adaptive  Personality  Assessment  System  (TAP AS).  A  preliminary  set  of 
cross  job  work  activity  descriptors,  customized  to  the  Anny,  were  developed  for  this  project  and 
for  the  0*NET  evaluation.  An  expanded  version  of  these  cross  job  descriptors  was  recently  used 
in  a  content  validation  of  the  ASVAB  (see  Waters  et  ah,  2009  and  Appendix  F). 

Lower-order  work  activities  are  excluded  from  the  list  of  descriptors  for  two  reasons. 
First,  the  time  and  financial  resources  required  to  generate  lower-order  work  activities  for  45-55 
MOS  could  be  tremendous.  Second,  collecting  data  on  those  descriptors  is  not  essential  for  the 
near-term  purpose  of  clustering  MOS  or  for  developing  (or  identifying)  end-of-training  criteria, 
based  on  higher-order  infonnation  about  the  MOS  sampled.  Once  the  clusters  have  been 
developed  and  the  20-30  priority  MOS  identified  for  criterion-related  validation  research,  lower- 
order  work  activities  could  then  be  formulated  for  those  targeted  MOS. 
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Activity  3:  Select  the  data  collection  method  and  from  whom  data  are  to  be  collected. 

As  addressed  in  the  preceding  section,  no  single  data  collection  method  is  likely  to  prove 
“best”  for  collecting  ratings  on  the  selected  domains.  However,  given  the  number  of  MOS  to  be 
sampled,  a  questionnaire-based  approach  is  likely  to  be  the  most  cost  affordable,  while  at  the 
same  time  likely  to  yield  sufficiently  reliable  and  valid  ratings.  This  is  particularly  true  since  the 
questionnaires  for  the  recommended  domains  have  already  been  developed  and  programmed  for 
web-based  administration  using  the  Army’s  InterForm  platfonn.  There  are  multiple  strategies  for 
enhancing  the  potential  of  the  existing  questionnaires  to  differentiate  across  MOS  that  would  not 
require  extensive  resources  from  the  Army  to  implement  (e.g.,  using  ratings  from  multiple 
domains  to  cluster  MOS,  using  data  for  a  larger  sample  of  MOS  to  eliminate  items  not  useful  for 
discriminating  across  MOS,  having  participants  select  the  top  five  most  important  and  least 
important  attributes). 

Another  practical  advantage  to  administering  web-based  questionnaires  is  that  the 
questionnaires  could  be  easily  administered  to  both  supervisors  and  incumbents  for  the  same 
administrative  cost(s).  Unless  collecting  data  from  incumbents  for  the  MOS  sampled  interferes 
with  other  Army  data  collection  priorities  or  identifying  incumbents  to  participate  requires 
significant  resources,  it  makes  sense  then  to  administer  the  questionnaires  to  raters  from  both 
groups.  Doing  so  would  enable  the  Army  to  more  comprehensively  examine  the  practical 
implications  of  collecting  ratings  from  the  two  groups  (e.g.,  by  comparing  what  differences,  if 
any,  exist  between  the  clustering  solutions  obtained  from  supervisor  and  incumbent  ratings). 

Activity  4:  Prepare  and  administer  the  data  collection  materials.  Modify  the  database. 

The  next  activity  would  involve  finalizing  the  questionnaires  for  web-based 
administration  and  modifying  the  existing  database,  as  needed.  Based  on  the  preceding 
discussion,  any  changes  to  the  questionnaires  would  generally  be  minimal  and  would  not  require 
generating  job-specific  content  (e.g.,  adding  an  additional  exercise  to  identify  the  top  five  most 
and  least  important  descriptors  within  each  domain).  The  results  of  the  field  test  indicate  that  20- 
25  participants  per  questionnaire,  on  average,  would  yield  sufficiently  reliable  ratings  to 
construct  group  means  for  clustering  the  MOS.  Assuming  a  5%  response  rate,  surveys  should  be 
sent  to  about  500  Soldiers  (supervisors  and  incumbents)  per  MOS. 

Activity  5:  Analyze  the  ratings  and  cluster  the  MOS. 

The  primary,  near-term  objective  of  this  activity  is  to  derive  clusters  to  facilitate  the 
selection  of  target  MOS  for  criterion-related  validation  research.  In  general,  clustering  the  MOS 
would  involve  two  steps.  The  first  step  consists  of  clustering  the  MOS  on  the  basis  of  the  ratings 
for  each  of  the  targeted  domains,  separately.  The  resulting  clusters  could  be  further  refined  (a) 
empirically  (i.e.,  from  component  analyses  of  applicable  descriptors  within  each  domain)  or  (b) 
rationally  on  the  basis  of  which  descriptors  carry  the  greatest  potential  for  classification  gains 
(e.g.,  specific  interests  first,  work  styles  second,  and  then  values).  The  second  step  would  be  to 
perform  a  series  of  two-stage  cluster  analyses,  where  MOS  are  clustered  first  on  the  basis  of  the 
higher-order  work  activities  and  then  on  each  worker-oriented  domain,  separately.  The  resulting 
two-stage  cluster  solutions  could  then  be  compared  empirically  to  determine  which  carries  the 
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greatest  potential  for  enhancing  the  Anny’s  existing  classification  system,  thereby  fonning  the 
basis  on  which  MOS  will  be  sampled  for  future  criterion-related  validation  research. 


Conclusion 

Developing  and  implementing  a  new  Army-wide  job  analysis  system  that  collects  critical 
information  without  being  unwieldy  to  maintain  is  a  daunting  but  necessary  task.  Implementing 
such  a  system,  however,  will  enable  the  Army  to  accomplish  its  near-  and  long-term 
classification  research  objectives.  A  system  exhibiting  the  following  features  is  most  likely  to 
meet  these  objectives: 

•  A  common  language,  customized  to  the  Army  context,  for  describing  similarities  and 
differences  in  MOS.  This  common  language  should  consist  of  a  reasonably 
comprehensive  set  of  descriptors  representing  targeted  work  (i.e.,  work  activities,  work  or 
job  context,  machine-tools-equipment-technology)  and  worker-oriented  (i.e.,  knowledge, 
skills,  abilities,  work  values,  interests)  domains.  These  descriptors  must  be  sufficient  for 
describing  any  MOS. 

•  Cross-MOS  descriptors  for  use  in  identifying  similarities  and  differences  across  MOS. 

•  Descriptors,  in  particular  work  activities  (i.e.,  what  Soldiers  do  on  a  job),  at  multiple 
levels  of  generality  that  can  be  organized  hierarchically  to  support  the  Army’s  needs  for 
job  infonnation  at  multiple  levels  of  aggregation. 

•  The  flexibility  to  combine  descriptors  or  data  on  descriptors  from  multiple,  traditionally 
independent  domains,  and  in  new  or  innovative  ways  that  enable  the  identification  of 
similarities  and  differences  in  Army  jobs. 

•  A  reasonable  cost  to  implement  the  system. 

•  Sufficient  flexibility  so  that  the  descriptors  can  be  refined  and  updated,  as  needed.  The 
system  needs  to  include  a  mechanism  for  adding  new  descriptors,  or  modifying  existing 
descriptors,  as  job  information  on  a  larger  sample  of  MOS  is  collected  or  MOS 
requirements  change  over  time. 

•  A  foundation  based  on  descriptor  taxonomies,  developed  or  refined  using  a  combined 
top-down  and  bottom-up  approach,  as  recommended  by  the  ACRP. 

•  A  relational  database  that  collects,  organizes,  and  stores  job  analysis  data  systematically 
over  time  and  that  facilitates  the  integration  of  these  data  with  other  relevant  personnel 
data  (e.g.,  criterion-related  validity  estimates). 

The  prototype  job  analysis  system  developed  and  field  tested  in  this  effort  was  designed 
with  these  features  in  mind.  The  results  of  the  field  test  demonstrate  that  the  prototype  system 
has  the  potential  to  reliably  and  validly  describe  entry-level  jobs  using  a  questionnaire-based 
approach  that  is  reasonably  cost-effective  to  implement  and  administer  Army-wide.  Further, 
many  of  the  recommend  follow-on  steps  are  intended  to  take  advantage  of  available  resources 
(e.g.,  0*NET)  while  incorporating  lessons  learned  during  the  prototype  field  test  and  experience 
with  the  Anny’s  current  system.  In  the  end,  it  is  important  to  recognize  that  there  is  arguably  a 
finite  number  of  ways  in  which  jobs  can  be  described.  A  new  system  may  not  seem  to  be 
dramatically  different  from  existing  ones  on  the  surface.  Nevertheless,  it  may  produce  different 
and  more  useful  results  in  practice.  Modifying  the  prototype  system  field  tested  here,  along  with 
paying  additional  attention  to  process-related  issues  (e.g.,  how  data  are  collected)  and 
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coordinating  efforts  with  existing  Army  systems  and  associated  stakeholders  (particularly 
TRADOC)  has  appreciable  potential  to  accomplish  the  goals  set  forth  by  ARI  and  articulated  in  a 
roadmap  formulated  by  the  advisory  panel  it  established  (Campbell  et  ah,  2007). 
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Appendix  A 


Recommendations  and  Specifications  for  Formulating  Army  Job  Descriptors  and  Metrics 
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Work  Activities,  to  include  Weapons-Equipment-Tools 


Definitions 

Work  activities  are  behaviorally  based  descriptions  of  what  a  Soldier  (or  Officer)  in  an 
occupation/job  does  (i.e.,  the  activities  s/he  performs),  and  is  held  accountable  to  perfonn,  over 
some  span  of  time  that  contributes  to,  and  creates,  value  to  the  Anny.  Generally  speaking,  there 
are  three  major  categories,  or  types,  of  work  activities 

Occupation/Job-Specific  Technical  Activities  reflect  work  activities  that  constitute  the 
core  technical  requirements  of  an  occupation/)' ob  and  that  differentiate  an  Army 
occupation/)' ob,  or  cluster  of  occupations/jobs,  from  others  (e.g.,  performing  operator 
maintenance  checks  and  services;  repairing  mechanical  systems;  installing  and 
assembling  electronic  components;  installing  wire  and  cables;  driving  a  light  wheeled 
vehicle). 

Non-Occupation/Job-Specific  (i.e.,  Army-Wide)  Technical  Activities  reflect  work 
activities  constituting  the  core  technical  requirements  that  are  not  unique  to  an 
occupation/)' ob  but  are  required  by  most  or  all  jobs  in  the  Army  (e.g.,  performing  basic 
first  aid;  navigating  in  the  field;  aiming  and  firing  a  weapon;  maintaining  a  weapon). 

Non-Technical  Activities  reflect  work  activities  that  are  non-technical,  and  typically 
intra-  and  inter-personal,  in  nature  (e.g.,  exhibiting  self-management,  exhibiting  self- 
directed  learning,  relating  to  and  supporting  peers,  demonstrating  peer  leadership, 
demonstrating  teamwork).  These  requirements  could  be  specific  to  an  occupation/)' ob,  or 
cluster  of  occupations/jobs.  Alternatively,  they  could  be  required  of  most  or  all  Anny 
jobs  (i.e.,  Army-wide). 

Each  of  these  categories  of  work  activities  could  be  defined  at  multiple  levels  of 
generality,  which  can  be  arranged  hierarchically  within  each  category  according  to  well-defined 
rules  to  be  established  as  part  of  Phase  1  of  this  pilot  project.  At  a  minimum,  we  plan  on 
formulating  work  activities  at  two  levels  of  generality,  as  outlined  below7 

Major  Duties  (MDs)  represent  the  highest-order  work  activities  and  are  cross-job  in 
nature.  They  describe  a  definable  and  nontrivial  duty,  or  responsibility,  for  which  a 
Soldier  (or  Officer)  is  accountable  to  perfonn,  which  has  stakeholders  to  whom  the 
results  (i.e.,  outputs)  are  important  and  meaningful,  and  which  entails  work  of  significant 
complexity  and  duration.  More  specifically,  MDs  describe  what  a  Soldier  (or  Officer) 
does,  and,  as  appropriate,  to  what  or  for  whom  it  is  done.  MDs  duties  are  typically  cross¬ 
job  in  nature  (i.e.,  these  requirements  are  relevant  across  multiple,  and  in  some  cases,  all, 
Anny  jobs).  They  are  behaviorally  based  and  represent  an  aggregation  of  multiple, 
related  lower-order  work  activities,  as  defined  below. 


7  Although  we  will  be  formulating  work  activities  at,  at  least,  two  levels  of  generality,  we  will  be  capturing 
information  at  three  levels.  Task  statements  and  other  existing  information  on  occupation/job-specific  work 
activities  (e.g.,  from  Soldier  Manuals)  that  served  as  source  materials  for  formulating  MDs  and  KWAs  will 
constitute  the  lowest  level  of  description. 
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The  following  are  examples  of  MDs 

•S  “Repair  mechanical  systems.”  (Occupation-Specific  Technical) 

•S  “Install  electronic  components.”  (Occupation-Specific  Technical) 

•S  “Pack  and  load  materials.”  (Occupation-Specific  Technical) 

•S  “Send  and  receive  radio  signals.”  (Occupation-Specific  Technical) 

S  “Drive  wheeled  vehicles.”  (Occupation-Specific  Technical) 

•S  “Write  and  deliver  presentations.”  (Occupation-Specific  Technical) 

•S  “Engage  in  hand-to-hand  combat.”  (Army-Wide  Technical) 

•S  “Navigate  in  the  field.”  (Army-Wide  Technical) 

•S  “Survive  in  the  field.”  (Army-Wide  Technical) 

•S  “Lead  a  team.”  (Army-Wide  Non-Technical) 

•S  “Support  and  advise  peers.”  (Army-Wide  Non-Technical) 

Key  Work  Activities  (KWAs)  represent  lower-order  work  activities.  They  are  statements 
(or  categories)  of  moderate  specificity  that  describe  a  series  or  clustering  of  related 
behaviors  a  Soldier  (or  Officer)  performs  to  achieve  a  specific  work  objective.  More 
specifically,  KWAs  describe  what  a  Soldier  (or  Officer)  does,  to  what  or  for  whom  it  is 
done,  and  potentially  for  what  purpose  it  is  done.  KWAs  are  occupation  or  job-specific, 
to  some  degree,  although  it  is  possible  that  two  or  more  occupations/jobs  may  share  the 
same  KWA.  These  activities  typically  reflect  technical  work  activities,  but  could 
encompass  non-technical  ones  as  well.  At  their  lowest-level,  KWAs  represent  an 
aggregation  of  two  or  more  related  tasks,  as  currently  found  in  the  Army  (e.g.,  Soldier 
Manuals). 

The  following  are  examples  of  KWAs 

S  “Select  and  prepare  firing  positions  for  individual  weapon.” 

S  “Construct  field  expedient  firing  aids  for  individual  weapon.” 

“Load,  unload,  and  correct  malfunctions  for  individual  weapon.” 

“Aim,  track,  and  fire  individual  weapon  at  targets.” 

As  evident  from  these  definitions,  these  two  levels  of  work  activities  are  conceptually 
similar  (i.e.,  both  reflect  work-related  behaviors)  so  that  statements  reflecting  the  same 
underlying  activity  across  levels  can  be  linked.  Thus,  both  MDs  and  KWAs  are  expected  to 
follow  a  similar  fonnat  and  structure,  as  summarized  in  the  proceeding  sections.  The  chief 
difference  between  the  two  is  one  of  breadth  or  scope.  Specifically,  whereas  MDs  are  broader 
(i.e.,  higher-order)  and  are  expected  to  be  applicable  across  a  relatively  wide  range  of  Army 
occupations/jobs,  KWAs  are  more  specific  and  often  applicable  only  to  a  subset  of  Anny 
occupations/jobs  requiring  a  similar  activity. 
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Objectives  and  Purposes  to  be  Served 


The  purposes  for  which  these  prototype  job  descriptors  are  intended  to  serve  were 
previously  outlined  (pp.  2-3;  10-12).  More  specifically,  MDs  are  primarily  intended  to  provide  a 
common  basis,  or  set  of  cross-job  descriptors,  on  which  all  Army  jobs  could  be  scaled/rated  for 
the  purposes  of  clustering  jobs  (i.e.,  to  guide  the  sampling  of  Army  jobs  for  criterion-related 
validation  studies;  to  generalize,  or  transport,  criterion-related  validity  estimates  and  predicted 
performance  equations  across  jobs;  to  identify  potential  candidates  for  “mid-range”  criterion 
measures).  MDs  could  also  be  used  to  elicit  infonnation  from  Army  SMEs  on  lower-order  work 
activities,  in  particular  non-technical  requirements,  in  situations  where  existing  information  on 
these  requirements  does  not  exist  or  is  otherwise  incomplete.  KWA  categories  and  statements, 
and  the  “optimal”  level(s)  at  which  each  is  to  be  specified,  are  intended  to  provide  information 
for  use  in  clustering  Army  jobs  (i.e.,  to  determine  the  limits  and  understand  the  implications  of 
generalizing,  or  transporting,  criterion-related  validity  estimates  and  predicted  perfonnance 
equations  across  jobs),  as  well  as  infonnation,  and  possibly  content,  for  developing  and/or 
refining  performance-related  criterion  measures  (i.e.,  ratings,  knowledge  tests). 

Recommended  Structure  and  Format  for  Major  Duties  (MDs) 

MDs  represent  statements  describing  a  definable  and  nontrivial  duty,  or  responsibility, 
for  which  a  Soldier  (or  Officer)  is  accountable  to  perfonn,  which  has  stakeholders  to  whom  the 
results  (i.e.,  outputs)  are  important  and  meaningful,  and  which  entails  work  of  significant 
complexity  and  duration.  The  following  are  recommendations  for  the  format  and  structure  of 
MDs  and  are  intended  to  serve  as  a  guide  when  generating  MD  statements. 

•  The  total  set  of  MDs  developed  should  be  sufficiently  comprehensive.  That  is,  it  should 
include  all  of  the  important  duties,  technical  and  non-technical,  and  not  simply  constitute  a 
sampling  of  representative  duties  required  of  Soldiers  (or  Officers). 

•  Statements  should  describe  MDs,  and  not  worker-related  characteristics.  MD  statements 
should  describe  a  set,  or  cluster,  or  related  work  activities  Soldiers  are  responsible  for 
performing.  They  should  not  describe  occupation-specific  knowledges  and  skills,  personal 
qualities,  or  other  worker-oriented  characteristics.  The  inclusion  of  these  requirements  in 
MDs  should  be  avoided. 


Install  electronic  components 
a/  Troubleshoot  mechanical  systems 

a/  Repair  individual  weapons 

a/  Lead  a  team 

X  Knowledge  of  mechanical  principles 

X  Understands  weapon  operating,  service  or  repair  manuals 
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•  Statements  should  reflect  important  duties  or  responsibilities  that  are  as  objective, 
obser-vable,  and  as  fully  specified  as  possible.  That  is,  statements  should  lend  themselves  to 
measurement,  either  on  an  order-of-magnitude  (quantitative)  basis,  or  in  tenns  of  clearly 
observable  and  understandable  criteria  by  which  the  presence  or  absence  of  the  described 
duty  or  responsibility  can  be  reliably  assessed  or  evaluated.  Doing  this  is  important  both  for 
purposes  of  collecting  job  analysis  data  on  these  descriptors,  as  well  as  developing  criterion 
measures. 

•  Statements  should  be  at  the  appropriate  level  of  generality.  MDs  should  not  be  so  general  as 
to  be  nearly  universally  applicable  (e.g.,  “Makes  Decisions,”  “Inputs  information”, 
“Identifies  objects,  actions,  and  events”,  “Repairs  equipment”).  Alternatively,  MDs 
statements  should  not  be  as  specific  as  KWAs,  as  defined  here.  MDs  should  be  sufficiently 
general  that  they  can  be  applied  across  Army  jobs  for  the  purposes  of  describing  similarities 
and  differences  in  these  jobs  using  a  common  metric. 


"V  Repair  electrical  systems 
X  Correct  malfunctions  of  alternator 
X  Correct  malfunction  of  batteries 

X  Replace  protective  control  box 


•  Statements  should  convey  a  complete  and  well-defined  MD,  making  clear  what  is  done,  and, 
as  appropriate,  to  what  or  for  whom  it  is  done.  A  MD  statement  should  include  both  a  verb 
(specifying  the  action,  e.g.,  “Repair”)  and  an  object  (a  noun  specifying  who  or  what  is  acted 
upon,  e.g.,  “weapons”,  “electronic  systems”,  “mechanical  systems”).  Avoid  multiple  verbs 
that  imply,  or  reflect,  more  than  one  type  of  duty  and,  potentially,  differing  work  activities. 
Similarly,  avoid  verbs  that  are  general  or  have  multiple  meanings  (e.g.,  Does  “operate 
weapon”  mean  “fire  weapon”  or  “clean”  or  “load”?). 

•  Verbs  conveying  a  concrete,  specifiable  duty  are  preferable  to  more  abstract  or  complex 
verbs,  which  often  leave  the  nature  of  the  duty  unclear  or  ambiguous.  For  example,  “explain” 
is  better  than  “consult”;  “check,”  “inspect,”  or  “review”  are  better  than  “ensure  that”;  other 
verbs  that  are  often  problematic  in  this  way  are  “coordinate”  and  “determine,”  which  are 
usually  unclear  as  to  the  nature  of  the  duty. 

•  Use  standardized  wording  structure  and  consisten  t  terminology  as  far  as  possible  across  the 
sets  of  MDs.  MD  statements  should  follow  the  same  wording  structure,  as  outlined  in  the  two 
preceding  bullets.  When  referencing  the  same  object  use  the  same  term  consistently  across 
statements.  For  example,  if  a  weapon  maintained  and  operated  by  an  individual  Soldier  (or 
Officer)  is  referred  to  as  an  “individual  weapon”,  that  tenn  should  be  used  consistently. 

Avoid  using  alternative  wording  or  terms  (e.g.,  “assigned  weapon”  or  “personal  weapon”)  for 
the  same  object  unless  truly  referencing  a  new  and  different  object. 
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•  Statements  should  be  as  simple  and  concise  as  possible,  without  sacrificing  meaning,  and  not 
make  use  of  occupation-specific  jargon  and  acronyms.  Compound  and  complex  (hyphenated, 
slashed)  word  constructions  (e.g.,  "Develop  alternative/modified  strategic/tactical  plans...") 
should  be  avoided.  If  the  hyphen  or  slash  means  “or,”  it  is  better  to  write  “or”;  if  it  means 
“and,”  two  statements  may  be  warranted.  Although  MD  statements  should  be  Army-specific, 
the  language  used  should  be  “user-friendly”  (i.e.,  free  of  jargon  or  complex  terminology 
specific  to  that  job  or  occupation).  That  is,  statements  should  be  simple  but  sufficiently 
descriptive  of  the  duty  performed  so  that  an  individual  without  specific  training  and/or  tenure 
in  the  Anny  can  understand  what  is  done,  and,  as  appropriate,  to  what  or  for  whom  it  is  done. 

•  Statements  should  be  descriptive  rather  than  evaluative.  For  example,  “Maintains  a  motor 
vehicle”  is  appropriate.  “Maintains  a  motor  vehicle  in  a  manner  that  minimizes  unneeded 
repairs  and  maintenance,”  is  not. 


V  Maintains  a  motor  vehicle 

X  Maintains  a  motor  vehicle  in  a  manner  that  minimizes  unneeded  repairs 
and  maintenance 


•  Statements  should  avoid  the  use  of  examples  (e.g.,  xxx)  to  clarify  the  boundaries  of  similar, 
and  potentially  related,  duties.  The  use  of  examples  for  purposes  of  delineating  differences 
across  similar,  and  potentially  related,  duties  should  be  avoided.  As  much  as  feasible, 
differences  across  MDs  should  be  reflected  in  the  statements  themselves. 


X  Repair  electrical  systems  (e.g.,  instrument  panels,  vehicle  starter  systems, 
batteries) 

X  Repair  mechanical  systems  (e.g.,  engines,  transmissions,  vehicle  brake 
systems) _ 


Proposed  Steps  and  Procedure  for  Generating  Major  Duties  (MDs) 

The  following  are  the  proposed  steps  and  procedure  for  generating  MDs.  At  the  end  of 
this  process  there  should  be  a  set  of  50-100  MD  statements  (roughly),  reflecting  both  technical 
and  non-technical  work  activities,  for  use  in  workshops  with  proponent  SMEs  to  formulate  and 
calibrate  a  hierarchical  work  activities  taxonomy  linking  these  higher-order  requirements  to 
KWAs,  as  well  as  the  rules  for  doing  so.  The  generation  of  MDs  will  take  place  at  the  same  time 
as  the  initial  drafting  of  KWAs,  as  described  in  the  proceeding  sections. 

Activity  1.  Starting  with  a  pool  of  candidate  performance  descriptors  customized  to  Army, 
the  Descriptor  Development  Team  generates  a  preliminary  taxonomy  (or  first  draft)  of  MDs  by 
reformatting  and  editing  these  descriptors  in  line  with  the  proposed  guidelines  and 
specifications,  as  outlined  in  this  document.  Completing  this  activity  involves  the  following  steps 
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Step  1.1:  Review  candidate  performance  descriptors  and  existing  military  performance 
taxonomies.  Candidates  to  start  from  include 

•  Army- wide  performance  dimensions  and  occupation/]  ob-specific  performance 
categories  from  the  Army21  projects  (PerformM21,  NC021,  Select21,  Anny  Class). 

•  Task  categories  and  job  activities  from  the  Anny  Task  Questionnaire  and  Job  Activity 
Questionnaire,  respectively,  from  Project  SYNVAL. 

•  Performance  dimensions  from  existing  military  perfonnance  models  developed  for 
the  Army  and  the  other  Services  (e.g.,  from  Project  A). 

•  Task  categories  from  relevant  Army  manuals  and  documents,  in  particular  those 
focused  on  Army-wide  technical  and  non-technical  duties  (e.g.,  Enlisted  Soldier  and 
Officer  Common  Task  Lists;  Leadership  Manual).8 

Step  1.2:  Create  an  Excel  Workbook  to  document  the  MD  development  process.  Creation 
of  the  Workbook  involves  importing  candidates  (from  Step  1.1)  determined  to  be  most  useful 
and  coding  the  selected  performance  descriptors  so  that  the  MDs  generated  can  be  tracked/linked 
back  to  the  relevant  source  material.  A  useful  strategy  for  narrowing  down,  or  otherwise  refining, 
the  initial  list  of  candidate  performance  descriptors  is  to  crosswalk  each  set  of  descriptors  against 
each  other,  eliminating  redundancies  as  needed. 

Step  1.3:  Divide  the  work  so  that  each  member  of  the  Descriptor  Development  Team 
focuses  on  and  is  responsible  for  one  of  the  three  major  categories,  or  types,  of  work  activities 
(i.e.,  Occupation-Specific  Technical;  Non-Occupation  Specific  or  Anny-Wide  Technical,  Non- 
Technical). 

Step  1.4:  Generate  a  first  draft  of  MDs,  reformatting  and  revising  the  candidate 
performance  descriptors  in  line  with  the  proposed  guidelines  and  specifications,  as  outlined  here, 
to  ensure  consistency  in  breadth,  scope,  and  level  of  generality.  As  part  of  this  step,  the 
Descriptor  Development  Team  should  make  an  effort  to  identify  Army-wide  technical  and  non¬ 
technical  MDs  potentially  missing  from  the  candidate  performance  descriptors  and/or  those 
requiring  refinement.  To  do  this,  first  reformat  and  revise  the  candidate  performance  descriptors 
already  identified.  Next,  cross-walk  this  preliminary  set  of  MDs  for  these  targeted  domains 
against  (a)  non-military,  civilian  work-related  activities  or  performance  taxonomies  (e.g., 
Cunningham’s  Occupational  Analysis  Inventory  [OAI],  0*NET  Generalized  Work  Activities 
[GWAs]  and  Detailed  Work  Activities  [DWAs])  and  (b)  existing  taxonomies  relevant  to  one  or 
more  targeted  performance  dimensions  (specifically,  organizational/peer  support,  teamwork,  and 
leadership)  from  the  1-0  literature.  Based  on  this  crosswalk,  add  MDs  to  the  preliminary 
taxonomy  and/or  revise  existing  ones,  as  needed. 

Step  1.5:  Prepare  the  MD  Workbook,  ensuring  that  the  MDs  and  documentation  on  all 
source  materials,  and  so  on,  are  included.  When  finished,  post  the  Workbook  to  the  designated 
folder  on  the  S-drive. 

Activity  2.  A  Descriptor  Review  Team  reviews  preliminary  taxonomy  of  MDs  to  ensure 


s  The  task  statements  from  these  manuals  are  likely  to  be  useful  for  refining  the  task  categories. 


A-7 


consistency  with  proposed  guidelines  and  specifications  and  that  the  taxonomy  provides 
sufficiently  comprehensive  coverage  of  the  performance  space.  Descriptor  Development  Team 
implements  recommended  edits,  as  needed. 

Activity  3.  Using  the  preliminary  taxonomy  of  MDs,  conduct  a  sorting  exercise  with  the 
Descriptor  Development  Team  comparable  to  that  proposed  for  the  SME  workshops  first 
outlined  in  the  ACRE  report  (pp.  38-39).  The  purpose  of  this  sorting  exercise  is  twofold:  (a)  to 
identify  additional  refinements  to  the  pool  of  Army-wide  and  non-technical  MDs  and  (b)  to 
identify  and  recommend  potential  candidates  for  elimination  so  that  the  pool  of  MDs  can  be 
reduced  to  about  50-100,  if  needed.  In  brief,  the  sorting  exercise  would  consist  of  the  following. 
First,  sort  the  MD  statements  for  the  targeted  domains  into  the  0*NET  GWAs  and  DWAs.  Next, 
sort  within  the  MD  statements,  maximizing  the  homogeneity  of  content  within  categories  and 
minimizing  overlap  between  categories.  Finally,  sort  the  drafted  KWA  categories  (from  2-3 
MOS)  into  MD  statements.  A  senior  Descriptor  Review  Team  member  would  facilitate  this 
sorting  exercise. 

Activity  4.  The  Expert  Advisory  Panel,  high-level  Army  SMEs  (e.g.,  members  of  ARE s 
Army  Testing  Program  Advisory  Team  [ATPAT]),  and/or  Army  job  analysts  review  the  proposed 
pool  of  MDs  and  select  metrics  (i.e.,  scales).  The  purpose  of  this  review  is  threefold:  (a)  to 
identify  and  recommend  modifications  and  edits  to  the  proposed  pool  of  MDs,  (b)  to  identify  and 
recommend  potential  candidates  for  elimination,  in  particular  any  occupation-specific  technical 
duties  that  are  no  longer  relevant  to  Army,  and  (c)  to  identify  and  recommend  possible  additions 
to  the  pool  of  occupation-specific  technical  duties  based  on  the  Anny’s  existing  occupational 
structure.  At  a  minimum,  the  metrics  would  include  scales  assessing  the  frequency  and 
importance  of  the  MD  to  an  occupation/job  (e.g.,  the  existing  0*NET  frequency  and  importance 
scales  used  when  rating  GWAs).  The  Descriptor  Development  Team  implements  edits,  as 
needed. 

Activity  5.  Conduct  workshops  with  Army  SMEs.  The  purpose  of  these  workshops  is  to 
collected  feedback  and  preliminary  data  on  MDs  from  Army  SMEs. 


Recommended  Structure  and  Format  for  Key  Work  Activities  (KWAs) 

As  discussed,  KWAs  are  categories  and  statements  describing  a  series  or  cluster  of 
related  behaviors  a  Soldier  (or  Officer)  perfonns  to  achieve  a  specific  work  objective.  The 
following  are  recommendations  for  the  format  and  structure  of  KWAs  and  are  intended  to  serve 
as  a  guide  when  generating  KWAs. 

•  The  total  set  of  KWAs  developed  should  be  sufficiently  comprehensive.  That  is,  the  set  should 
include  all  of  the  important  KWAs,  technical  and  non-technical,  and  not  simply  constitute  a 
sampling  of  representative  KWAs  required  of  Soldiers  (or  Officers). 
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•  Statements  should  describe  KWAs  and  not  worker-oriented  characteristics.  KWAs  should 
describe  the  work-related  behaviors  Soldiers  (or  Officers)  must  perform.  KWAs  should  not 
describe  occupation-specific  knowledges  and  skills,  personal  qualities,  or  other  worker- 
oriented  characteristics.  The  inclusion  of  these  requirements  in  KWAs  should  be  avoided. 

For  example,  the  Anny’s  Soldier’s  Manual  of  Common  Tasks  (STP  21-1-SMCT) 
(Department  of  the  Army,  2006)  includes  the  task:  “Comply  with  the  Law  of  War  and  the 
Geneva  and  Hague  Conventions.”  In  part,  “Law  of  War  and  the  Geneva  and  Hague 
Conventions”  reflects  an  Army-wide  knowledge  required  for  the  successful  performance  of 
multiple  tasks  (e.g.,  treatment  of  prisoners,  etc.).  When  generating  KWAs,  a  separate  listing 
of  non-work  activities  should  be  made  for  future  use  in  formulating  relevant  worker-oriented 
characteristics. 

•  Statements  should  reflect  importan  t  KWAs  that  are  as  objective,  observable,  and  as  fully 
specified  as  possible.  That  is,  statements  should  lend  themselves  to  measurement,  either  on 
an  order-of-magnitude  (quantitative)  basis,  or  in  terms  of  clearly  observable  and 
understandable  criteria  by  which  the  presence  or  absence  of  the  described  KWA  can  be 
reliably  assessed  or  evaluated.  Doing  this  is  important  both  for  purposes  of  collecting  job 
analysis  data  on  these  descriptors,  as  well  as  developing  criterion  measures. 

•  Use  behavioral  homogeneity  as  a  guide  when  generating  KWAs  to  ensure  that  statements  are 
at  an  appropriate  level  of  generality  (or  specificity).  KWAs  should  be  slightly  broader  and 
more  general  than  specific  task  statements,  as  currently  found  in  the  Anny  (e.g.,  Soldier 
Manuals,  Officer  Foundation  Standards).  A  KWA  should  be  written  to  encompass  tasks  or 
behaviors  that  collectively  represent  essentially  the  same  activity,  even  though  the  type  of 
equipment  involved  or  the  specific  knowledge  required  could  (and  do)  vary.  That  is,  a  KWA 
statement  should  represent  a  set,  or  cluster,  of  related  behaviors  reflecting  the  same 
underlying  work  activities.  In  the  example  below,  the  last  statement  includes  too  many 
heterogeneous  behaviors,  each  reflecting  different  work  activities.  Specifically,  “engage 
targets”  refers  to  aiming  and  firing  the  weapon  -  an  activity  that  typically  takes  place  on  the 
battlefield  in  support  of  offensive  and/or  defensive  operations.  “Maintaining  a  weapon  (e.g., 
M-9  pistol”  involves  disassembling/assembling,  cleaning,  and  lubricating  parts  -  an  activity 
that  takes  place  off  the  battlefield. 


V  Maintain  individual  weapon 

a/  Aim,  track,  and  fire  individual  weapon  at  target 

a/  Load,  unload  and  correct  malfunctions  in  individual  weapon 

X  Load,  unload,  engage  targets,  reduce  stoppages,  and  maintain  an  M9  pistol 


Statements  should  avoid  reference  to  specific  weapons-equipment-tools  (WETs).  Those 
experienced  with  Army  task  statements  know  that  the  Soldier  Manuals  and  training  are  not 
written  in  a  manner  that  generalizes  across  specific  pieces  of  equipment.  Instead,  there  are 
typically  separate  tasks  in  the  Soldier’s  Manual  representing  every  possible  combination  of 
activities  (e.g.,  Load/unload)  and  specific  WETs  (e.g.,  M9  pistol,  M16  Series  Rifle).  Our 
experience  is  that  Army  Subject  Matter  Experts  (SMEs)  sometimes  react  negatively  to  work 
activity  statements  that  are  devoid  of  WETs.  To  address  this,  KWAs  should  be  written 
without  making  reference  to  specific  WETs  unless  the  specific  WET  is  so  integral  to  the 
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work  activity  that  it  must  remain  in  the  KWA  (e.g.,  reading  a  map).  If  that  is  not  the  case  and 
there  are  multiple  task  statements  all  sharing  essentially  the  same  underlying  work  activity 
but  each  time  referencing  a  different  specific  WET  (e.g.,  different  guns  or  weapons),  make  a 
separate  list  of  specific  WET(s)  so  that  they  can  be  cross  referenced  with  the  relevant  KWAs 
later  on.  An  example  of  this  cross  referencing  appears  on  the  next  page  (see  Table  4). 
Eventually,  the  X’s  in  the  table  might  be  replaced  with  SME  judgments  about  the  frequency 
of  the  KWA  by  WET  combination.  In  the  KWA  statement  itself  the  specific  WET  should  be 
referenced  according  to  a  WET  categorization 

scheme  to  be  defined  (e.g.,  M9  pistol  might  be  referenced  as  a  “gun”;  an  Ml-Abrams  Tank 
as  a  “track  vehicle”). 

•  Statements  should  describe  a  complete  and  well-defined  activity,  making  clear  what  is  done, 
and,  as  appropriate,  to  what  or  for  whom  it  is  done.  Each  KWA  statement  should  start  with  a 
clear  action  verb  followed  by  a  description  of  an  object  (a  noun)  specifying  what  or  who  is 
acted  upon  (e.g.,  write  short  memos,  fire  individual  weapon,  inspect  electrical  system). 
Consistent  with  an  earlier  recommendation,  avoid  multiple  verbs  that  imply,  or  reflect,  more 
than  one  type  of  activity  and,  potentially,  differing  work  activities.  Similarly,  avoid  verbs  that 
are  general  or  have  multiple  meanings  (e.g.,  Does  “operate  weapon”  mean  “fire  weapon”  or 
“clean”  or  “load”?).  Verbs  conveying  a  concrete,  definable  activity  are  preferable  to  more 
abstract  or  complex  verbs,  which  often  leave  the  nature  of  the  activity  unclear  or  ambiguous. 
For  example,  “explain”  is  better  than  “consult”;  “check,”  “inspect,”  or  “review”  are  better 
than  “ensure  that”;  other  verbs  that  are  often  problematic  in  this  way  are  “coordinate”  and 
“determine,”  which  are  usually  unclear  as  to  the  nature  of  the  activity. 

•  Statements  should  use  context  (e.g.,  a  clause  specifying  purpose,  such  as  “to  identify 
business  problems  and  communications  needs  ”)  to  clarify  the  behavior(s)  when  complex  or 
vague  verbs  are  unavoidable.  For  example,  “Analyzes  information  on  the  situation  to 
determine  necessary  changes  in  combat  plans”  is  better  than  “Analyzes  situational 
information.”  That  is,  statements  should  not  only  make  clear  what  is  done,  and,  as 
appropriate,  to  what  or  for  whom  it  is  done,  but  for  what  purpose  it  is  done  as  needed. 
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KWAs  for  the  KWA  Category:  Operate  individual 

WEAPONS  (PISTOLS,  RIFLES,  MACHINE  GUNS,  AND  GRENADE 
_ LAUNCHERS) _ 

Select  and  prepare  firing  positions  for  individual  weapons 

Mount  and  dismount  sights  and  lights  on  individual  weapon 

Mount  and  dismount  machine  guns  on  tripods 

Construct  field-expedient  firing  aids  for  individual  weapon 

Load,  unload,  and  correct  malfunctions  for  individual  weapon 

Prepare  a  range  card  for  a  machine  gun 

Aim,  track,  and  fire  individual  weapon  at  targets 


-Equipment-Tools  (WETs) 


Weapons-Equipment-Tools 
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Aiming  Light  AN/PEQ-2A-Series 


•  Use  standardized  wording  structure  and  consistent  terminology.  KWA  statements  should 
follow  the  same  wording  structure,  as  outlined  in  the  two  preceding  bullets.  When 
referencing  the  same  object  use  the  same  term  consistently  throughout.  For  example,  if  a 
weapon  maintained  and  operated  by  an  individual  Soldier  (or  Officer)  is  referred  to  as  an 
“individual  weapon”,  that  tenn  should  be  used  consistently.  Avoid  using  alternative  wording 
or  tenns  (e.g.,  “assigned  weapon”  or  “personal  weapon”)  for  the  same  object  unless  truly 
referencing  a  new  and  different  object. 

•  Statements  should  be  as  simple  and  concise  as  possible,  without  sacrificing  meaning,  and 
should  not  make  use  of  occupation-specific  jargon  and  acronyms.  When  writing  KWA 
statements,  compound  and  complex  (hyphenated,  slashed)  word  constructions  (e.g., 

“Develop  alternative/modified  strategic/tactical  plans...”)  should  be  avoided.  If  the  hyphen  or 
slash  means  “or,”  it  is  better  to  write  “or”;  if  it  means  “and,”  two  statements  may  be 
warranted.  Although  the  activity  described  in  the  KWA  statements  should  be  Army-specific, 
the  language  used  should  be  “user-friendly”  (i.e.,  free  of  jargon  or  complex  terminology 
specific  to  that  job  or  occupation).  That  is,  statements  should  be  simple  but  sufficiently 
descriptive  of  the  activity  performed  such  that  an  individual  without  specific  training  and/or 
tenure  in  the  job  or  occupation  can  understand  the  requirement.  For  example,  in  the  Soldier 
Common  Task  manual  there  is  a  task  titled  “Perform  voice  communications.”  If  one  reads 
the  task  description,  the  actual  activity  consists  of  sending  a  short  radio  message. 


a/  Send  a  short  radio  message 
X  Perform  voice  communications 


•  Statements  should  be  descriptive  rather  than  evaluative.  KWAs  should  be  descriptive,  not 
prescriptive  or  evaluative.  For  example,  “Troubleshoot  and  correct  malfunctions  in  fuel 
system”  is  appropriate.  “Troubleshoot  and  correct  malfunctions  in  fuel  system  in  a  manner 
that  minimizes  unneeded  repairs  and  maintenance,”  is  not. 


Troubleshoot  and  correct  malfunctions  in  fuel  system 
X  Troubleshoot  and  correct  malfunctions  in  fuel  system  in  a  manner  that 
_ minimizes  unneeded  repairs  and  maintenance _ 


•  Statements  should  avoid  the  use  of  examples  (e.g.,  xxx)  to  clarify  the  boundaries  of  similar 
activities.  During  the  drafting  of  KWAs,  the  use  of  examples  for  purposes  of  delineating  the 
differences  across  similar  activities  should  be  avoided.  SMEs  tend  to  pay  a  lot  of  attention  to 
the  “e.g.,s.”,  shifting  their  focus  away  from  the  KWA.  As  much  as  feasible,  differences 
across  KWAs  should  be  reflected  in  the  statements  themselves. 


"k  Perform  basic  first  aid 

X  Perform  basic  first  aid  (e.g.,  CPR,  shock  prevention,  clear  throat  of 
casualty) 

X  Perform  first  aid  for  injuries  to  extremities  or  limbs  (e.g.,  put  on  field 
dressing,  tourniquet,  splint) _ 
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Proposed  Steps  and  Procedure  for  Generating  KWAs 

The  following  are  the  proposed  steps  and  procedure  for  generating  KWAs.  At  the  end  of 
this  process  there  should  be  a  set  of  KWA  statements  for  4-6  MOS,  reflecting  both  technical  and 
non-technical  work  activities,  for  use  in  workshops  with  proponent  SMEs  to  formulate  and 
calibrate  a  hierarchical  work  activities  taxonomy  linking  these  lower-order  requirements  to  MDs, 
as  well  as  the  rules  for  doing  so.  The  generation  of  KWAs  (for  an  initial  2-3  MOS)  will  take 
place  at  the  same  time  as  the  initial  drafting  of  MDs,  as  described  in  the  preceding  sections. 

Activity  1.  The  Descriptor  Development  Team  drafts  KWA  statements  for  each  of  the 
selected  MOS  in  line  with  the  recommended  guidelines  and  specifications .  Completing  this 
activity  for  each  MOS  involves  the  following  steps 

Step  1.1:  Review  available  information  on  the  MOS.  Candidates  to  start  from  include 

•  The  performance  categories  and  lower-order  work  activities  from  the  Army2 1 
projects  (PerformM21,  Select21,  Anny  Class). 

•  Existing  task  categories  and  task  statements  for  the  selected  MOS  from  relevant 
Army  manuals  and  publications,  in  particular  the  most  recent  Soldier  Manual  and 
Trainer’s  Guide  (STPs)  for  Enlisted  and  Officer  Foundation  Standards  (OFS)  for 
Officers. 

•  Existing  lists  of  task  statements  for  the  selected  MOS  used  in  a  recent  job  analysis 
survey  by  the  ARI’s  Occupational  Analysis  (OA)  Program. 

Information  on  candidate  descriptors  can  be  found  on  HumRRO’s  S-drive  at: 

S:\Project\Army  ONETYFirst  Cycle  Descriptor  DevelopmentYResources  and  Tools 

Step  1.2:  Create  an  Excel  Workbook  to  document  the  KWA  development  process  for  that 
MOS.  Creation  of  the  Workbook  involves  importing  existing  information  from  Step  1.1  (e.g., 
task  statements  from  the  STP)  and  then  coding  the  selected  descriptors  so  that  the  KWAs 
generated  can  be  tracked/linked  back  to  the  relevant  source  material. 

Step  1.3:  Once  this  information  has  been  imported  and  properly  coded,  create  a 
spreadsheet  within  the  Workbook  for  use  in  drafting  KWAs.  Label  the  spreadsheet,  “KWAs.” 

Step  1.4:  Sort  tasks  and  other  source  material  into  preliminary  categories.  These  initial 
categories  (or  first  cut)  should  be  fairly  broad  and  are  intended  to  help  facilitate  the  creation  and 
labeling  of  KWA  categories  (next  step). 

Step  1.5:  Starting  with  the  preliminary  categories  from  Step  1.3,  generate  and  label  KWA 
categories,  using  behavioral  homogeneity  as  a  guide.  This  is  expected  to  be  an  iterative  process, 
requiring  multiple  iterations  of  sorting  and  resorting,  each  time  making  progressively  finer 
distinctions  and  refinements  to  categories  previously  generated. 
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Step  1.6:  For  each  KWA  category  (from  Step  1.5),  generate  KWA  statements  that 
encompass  two  or  more  tasks  sharing  the  same  underlying  work  activity,  following  the 
guidelines  and  specifications  as  outlined  in  this  Blueprint. 

Step  1.7:  During  this  process,  create  a  separate  listing  of  specific  WETs.  This  will 
involve  first  detennining  whether  the  specific  WET  is  (a)  so  integral  to  the  work  activity  that  it 
must  remain  in  the  KWA  OR  (b)  should  be  extracted  or  summarized  at  a  broader  level  because 
there  are  multiple  requirements  (e.g.,  task  statements)  all  sharing  essentially  the  same  underlying 
work  activity,  but  each  time  referencing  a  different  specific  WET  (e.g.,  different  guns  or 
weapons).  If  (a),  retain  the  KWA  “as  is.”  If  (b),  list  the  specific  WETs  on  a  separate  spreadsheet 
within  the  KWA  Workbook  labeled  “WETs.”  To  facilitate  the  organization  of  the  WETs,  refer  to 
the  higher-order  WET  categories  from  the  Master  WETs  Workbook,  sorting  the  specific  WET 
into  the  category  that  best  describes  it.  Should  it  be  needed,  feel  free  to  create  new  subcategories 
that  describe  related  WETs  at  a  lower  level  of  detail  than  the  higher-order  categories  in  the 
Master  WETs  Workbook.  Use  this  newly  created  subcategory  or  the  higher-level  category  in  the 
KWA  statement. 

Step  1.8:  Identify  and  generate  non-technical  KWA  statements  potentially  missing  from 
the  preliminary  KWA  taxonomy  using  relevant  source  materials  containing  infonnation  on  non¬ 
technical  work  activities  (e.g.,  Mission  Training  Plans  [ARTEPS]).  As  before,  import  source 
materials  detennined  to  be  useful  into  the  Workbook,  placing  them  in  a  separate  spreadsheet. 
When  reviewing  the  source  materials,  think  about  the  non-technical  requirements  that  are  either 
(a)  implied  or  (b)  specifically  stated  in  these  materials.  Should  it  be  helpful,  or  should  no  existing 
source  materials  specific  to  the  selected  MOS  be  available,  review  and  use  the  0*NET  Work 
Context  Questionnaire  as  a  stimulus.  Be  sure  to  also  review  the  existing  KWAs  and  technical 
task  statements  (from  Task  1.7),  as  possible  non-technical  requirements  could  be  implied  in  one 
or  more  of  these  statements. 

Step  1.9:  With  the  non-technical  KWA  statements  (from  Step  1.8),  create  non-technical 
KWA  categories,  again  using  behavioral  homogeneity  as  a  guide.  Note  this  process  is  the 
opposite  of  the  process  for  generating  technical  KWAs,  and  that  is  by  design. 

Step  1.10:  As  with  the  WETs,  when  generating  KWA  statements,  make  note  of 
requirements  that  are  really  occupation-specific  KSs,  and  not  work  activities.  Maintain  a  list  of 
these  on  a  separate  spreadsheet.  This  information  could  be  useful  for  generating  occupation- 
specific  KSs.  Do  not  incorporate  KSs  into  a  KWA. 

Step  1.11:  Review  the  Workbook,  making  sure  that  everything  is  properly  coded  and 
labeled.  In  particular,  the  final  taxonomy  of  KWA  categories  and  statements  should  each  have  a 
code  associated  with  it.  For  KWA  categories,  the  code  should  be  as  follows 

MOS,  C  (for  “category”),  a  hyphen,  then  a  two  digit  identifier,  e.g., 

11BC-01 

11BC-02 

11BC-03... 
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For  KWA  statements,  the  code  should  be  as  follows 

MOS,  a  hyphen,  the  2  digit  KWA  category  code  (from  above),  a  hyphen,  then  a  three 
digit  identifier,  e.g., 

11B-01-001 

1  IB-01-002 

1  IB-01-003 

When  finished,  post  the  Workbook  containing  the  KWAs,  documentation  on  all  source 
materials,  and  so  on,  to  the  designated  folder  on  the  S-drive. 

Activity  2.  A  Descriptor  Review  Team  reviews  the  draft  KWA  statements  to  ensure 
consistency  with  proposed  guidelines  and  specifications  and  that  the  statements  provide 
sufficient  and  reasonably  comprehensive  coverage  of  the  performance  space.  Descriptor 
Development  Team  implements  recommended  edits,  as  needed. 

Activity  3.  Conduct  a  sorting  exercise  comparable  to  that  proposed  for  the  SME 
workshops  first  outlined  in  theACRP  report  (pp.  38-39)  with  the  Descriptor  Development  Team 
using  the  draft  KWA  statements  for  each  MOS.  The  purpose  of  this  sorting  exercise  is  twofold: 
(a)  to  identify  and  recommend  possible  modifications  and  refinements  to  the  draft  KWA 
statements  for  the  selected  MOS  and  (b)  to  generate  a  preliminary  sorting,  or  multiple  sortings 
reflecting  differing  levels  of  generality  (or  aggregation),  of  KWAs  for  Army  SMEs  to  react  to  in 
the  planned  workshops.  In  brief,  this  sorting  exercise  will  involve  the  following.  First,  sort  the 
KWA  statements  into  the  preliminary  taxonomy  of  MDs  previously  developed.  Next,  sort  within 
the  KWA  statements,  placing  statements  into  categories  that  maximize  homogeneity  within  a 
category  and  minimize  overlap  across  categories.  A  senior  member  of  the  Descriptor  Review 
Team  will  facilitate  the  sorting  exercise. 

Activity  4.  The  Expert  Advisory  Panel,  Army  SMEs  for  that  job  (if  available),  and/or 
Army  job  analysts  review  the  proposed  pool  of  KWA  statements,  WETs,  and  select  metric(s).  The 
purpose  of  this  review  is  threefold:  (a)  to  identify  and  recommend  modifications  and  edits  to  the 
draft  KWA  statements;  (b)  to  identify  and  recommend  potential  candidates  for  elimination,  if 
needed;  and  (c)  to  select  metric(s)  against  which  jobs  can  be  rated  on  the  draft  KWA  statements. 
At  a  minimum,  the  metrics  would  include  scales  assessing  the  frequency  and  importance  of  each 
KWA  to  an  MOS  (e.g.,  the  existing  0*NET  frequency  and  importance  scales  used  when  rating 
GWAs).  The  Descriptor  Development  Team  implements  recommended  edits,  as  needed. 

Activity  5.  Conduct  workshops  with  Army  SMEs  from  the  targeted  jobs.  The  purpose  of 
these  workshops  is  to  collected  feedback  and  preliminary  data  on  the  KWAs  from  Army  SMEs. 

Recommendations  and  Specifications  on  Work  Activity  Metrics 

Accompanying  the  work  activity  descriptors  are  metrics  (i.e.,  scales)  by  which  Anny 
occupations/jobs  could  be  assessed  on  these  same  descriptors.  At  a  minimum,  these  metrics 
should  include  generic,  Likert-type  scales  measuring  (a)  the  frequency  with  which  a  MD  or 
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KWA  is  performed  on  the  occupation/)' ob  and  (b)  the  importance  of  the  MD  or  KWA  to  the 
performance  of  the  occupation/job.  In  particular,  using  the  same,  existing  0*NET  and  Army  job 
analysis  scales,  if  available,  of  these  dimensions  (e.g.,  for  the  0*NET  GWAs)  would  enable  and 
facilitate  comparisons  between  the  prototype  job  descriptors  and  existing  alternatives. 

Occupation-Specific  Knowledges  and  Skills  (KSs) 


Definitions 

Occupation-specific  knowledges  and  skills  (KSs)  are  descriptions  of  the  developed 
knowledges  and  skills  a  Soldier  (or  Officer)  needs  to  perfonn  the  core  duties  and  work  activities, 
technical  and  non-technical,  required  of  an  Army  occupation  or  job,  or  cluster  of  occupations  or 
jobs.  In  contrast  to  aptitudes,  abilities,  and  other  worker  characteristics,  occupation-specific  KSs 
are  acquired  through  some  combination  of  education,  training  (on  and  off  the  job),  and 
experience  (i.e.,  they  are  trainable,  to  some  degree).  As  implied  by  their  definition,  occupation- 
specific  KSs  are  more  occupation-specific  than  basic  workplace  and  cross-functional  KSs  (e.g., 
basic  math  facility,  critical  thinking,  oral  and  nonverbal  comprehension,  problem-solving, 
reading  comprehension,  social  perceptiveness,  written  communication)  which  are  required  to  at 
least  some  degree  in  a  wide  range  of,  if  not  virtually  all,  occupations/jobs. 

As  with  work  activities,  occupation-specific  KSs  could  be  defined  at  multiple  levels  of 
generality,  arranged  hierarchically  according  to  well-defined  rules.  At  a  minimum,  these 
requirements  could  be  defined  at,  at  least,  two  levels  of  generality,  as  outlined  below 

Content  Knowledges  (CKs)  represent  labels  and  statements  that  describe  a  discrete  and 
integrated  body  of  declarative  knowledge  (e.g.,  subject  area,  discipline,  trade,  science)  a 
Soldier  (or  Officer)  must  know  to  perform  the  core  duties  and  work  activities  required  of 
an  Army  occupation/)' ob,  or  cluster  of  occupations/jobs.  CKs  facilitate  the  learning  and 
acquisition  of  more  specialized,  and  more  process-oriented  (i.e.,  procedural),  knowledges 
and  skills.  Thus,  CKs  provide  the  basis  for  process-oriented  KSs,  as  defined  here  (i.e., 
process-oriented  KSs  could  be  nested  in,  at  some  level,  CKs).  CKs  are  not  intended  to 
represent  prerequisite  knowledges  that  Soldiers  must  possess  at  the  start  of  their  technical 
training  (e.g.,  AIT,  OSUT).  Rather,  CKs  represent  knowledges  that  Soldiers  develop  and 
acquire  as  part  of  their  technical  training.  CKs  are  typically  technical  in  nature,  but  they 
could  encompass  non-technical  (i.e.,  intra-  and  inter-personal)  requirements  as  well.  CKs 
could  be  defined  at  multiple  levels  of  generality,  independent  of  PKSs. 

The  following  are  examples  of  CKs 

C  Computers  and  Electronics:  Knowledge  of  electric  circuit  boards,  processors, 
chips,  and  computer  hardware  and  software. 

C  Geography:  Knowledge  of  various  methods  for  describing  the  location  and 
distribution  of  land,  sea,  and  air  masses,  including  their  physical  locations, 
relationships,  and  characteristics. 

C  Medicine  and  Dentistry:  Knowledge  of  the  infonnation  and  techniques  needed 
to  diagnose  and  treat  injuries,  diseases,  and  deformities. 
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Process-Oriented  (Procedural)  Knowledges  and  Skills  ( PKSs )  represent  statements  that 
describe  the  knowledges  and  skills  of  the  processes,  operations,  and  procedures  needed  to 
perform  the  core  duties  and  key  work  activities  required  of  an  Army  occupation/job,  or 
cluster  of  occupations/jobs.  That  is,  PKSs  describe  a  Soldier’s  (or  Officer’s)  knowledge 
of  what  to  do  and  how  to  do  it.  PKSs  facilitate  the  performance  of  the  core  duties  and  key 
work  activities  a  Soldier  (or  Officer)  must  perform  in  their  occupation/)' ob.  Thus,  they  are 
more  proximal  than  CKs  to  what  a  Soldier  does,  and,  as  appropriate,  to  what  or  to  whom 
it  is  done.  As  with  CKs,  Process  (Operational)  KSs  are  typically  technical  in  nature,  but 
they  could  encompass  non-technical  (i.e.,  intra-  and  inter-personal)  requirements  as  well. 
For  example,  a  Soldier  serving  as  a  platoon  leader  would  require  PKSs  of  the  team 
activities  related  to  different  mission  types,  the  tactics  for  executing  those  activities,  how 
to  modify  and  improvise  on  those  tactics  as  needed,  and  so  on. 

In  sum,  whereas  CKs  are  higher-order  and  facilitate  the  learning  and  acquisition  of  PKSs, 
PKSs  are  lower-order  and  more  proximal  to  the  Soldier’s  (or  Officer’s)  work  activities.  At 
present,  only  CK  descriptors  will  be  developed,  although  PKSs  could  be  added  later. 

Objectives  and  Purposes  to  be  Served 

CKs  are  intended  to  provide  a  common  basis,  or  set  of  cross-descriptors,  on  which  Army 
jobs  could  be  rated  or  scaled  for  purposes  of  clustering  jobs  (i.e.,  to  guide  the  sampling  of  Anny 
jobs  for  criterion-related  validation  studies;  to  generalize,  or  transport,  criterion-related  validity 
estimates  and  predicted  perfonnance  equations  across  jobs;  to  identify  potential  candidates  for 
“mid-range”  criterion  measures,  in  particular  knowledge  tests). 

Recommended  Structure  and  Format  for  Content  Knowledges  (CKs) 

As  discussed,  CKs  represent  labels  and  statements  describing  the  developed  capacities  a 
Soldier  (or  Officer)  needs  to  perform  the  core  duties  and  work  activities,  technical  and  non¬ 
technical,  required  of  their  Anny  occupation/job.  The  following  are  recommendations  for  the 
format  and  structure  of  Content  Ks  and  are  intended  to  serve  as  a  guide  in  their  generation. 

•  The  total  set  of  CKs  developed  should  be  sufficiently  comprehensive.  That  is,  the  set  should 
include  all  of  the  important  CKs  and  not  simply  constitute  a  sampling  of  representative  CKs 
required  of  Soldiers  (or  Officers). 

•  Statements  should  describe  CKs  and  not  work-related  requirements  and  other  worker 
characteristics.  CKs  should  describe  the  declarative  knowledges  needed  by  Soldiers  (or 
Officers)  to  perform  the  core  duties  and  work  activities  required  of  their  Army 
occupation/job.  CKs  should  not  describe  work-related  requirements  (e.g.,  work  activities)  or 
other  worker-oriented  characteristics  (e.g.,  interests/values/temperament).  In  particular,  CKs 
should  not  be  a  translation  of  a  KWA  into  an  occupation-specific  KS  statement. 

•  Statements  should  reflect  important  CKs  that  are  as  objective,  observable,  and  as  fully 
specified  as  possible.  That  is,  CK  statements  should  lend  themselves  to  measurement,  either 
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•  Statements  should  describe  a  complete  and  well-defined  CK,  making  clear  what  is  known , 
and,  as  appropriate,  for  what  purpose  it  is  known  or  understood.  CK  statements  should  start 
with  the  clause  “knowledge  of,”  followed  by  an  object  (noun),  or  set  of  objects  (nouns), 
describing  what  is  known.  Avoid  multiple  nouns,  or  sets  of  nouns,  which  imply,  or  reflect, 
more  than  a  single,  well-defined  body  of  declarative  knowledge  (e.g.,  subject  area)  needed 
for  performance. 


V  Knowledge  of  basic  electronics  concepts  and  principles 
a/  Knowledge  of  basic  medical  terminology  and  symbols 
X  Knowledge  of  science 

X  Knowledge  of  the  proper  clothing  to  wear  and  when 


•  Statements  should  avoid  reference  to  specific  weapons,  equipment,  and  tools.  CK  statements 
should  be  written  without  making  reference  to  specific  WETs.  CKs  are  intended  to  reflect  a 
higher-level  of  generality  than  specific  WETs. 

•  Use  standardized  wording  structure  and  consistent  terminology.  CK  statements  should 
follow  the  same  wording  structure,  as  outlined  in  the  preceding  bullets.  When  referencing  the 
same  object  use  the  same  term  consistently  throughout. 

•  Statements  should  be  as  simple  and  concise  as  possible,  without  sacrificing  meaning.  When 
writing  CK  statements,  compound  and  complex  (hyphenated,  slashed)  word  constructions 
(e.g.,  “Understands  weapon  operation/service/repair  manuals...”)  should  be  avoided.  If  the 
hyphen  or  slash  means  “or,”  it  is  better  to  write  “or.”  Alternatively,  if  it  means  “and,”  two 
statements  may  be  warranted. 

•  Statements  should  be  descriptive  rather  than  evaluative.  CKs  should  be  descriptive,  not 
prescriptive  or  evaluative. 

•  Statements  should  avoid  the  use  of  examples  (e.g.,  xxx)  to  clarify  the  boundaries  of  similar 
CKs.  During  the  drafting  of  CK  statements,  the  use  of  examples  for  purposes  of  delineating 
the  differences  across  similar,  and  potentially  related,  CKs  should  be  avoided.  SMEs  tend  to 
pay  a  lot  of  attention  to  the  “e.g.,s.”,  shifting  their  focus  away  from  the  stem  of  the  CK 
statement.  As  much  as  feasible,  differences  across  CK  statements  should  be  reflected  in  the 
statements  themselves. 
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Proposed  Steps  and  Procedure  for  Generating  Content  Knowledges  (CKs) 


The  following  are  the  proposed  steps  and  procedure  for  generating  CK  labels  and 
statements.  The  output  of  this  process  will  be  a  set  of  CKs  for  use  in  workshops  with  proponent 
SMEs. 


Activity  1.  Starting  with  a  pool  of  candidate  descriptors,  the  Descriptor  Development 
Team  generates  a  preliminary  taxonomy  of  CKs.  Completing  this  activity  involves  the  following 
steps 


Step  1.1:  Review  candidate  CK  descriptors  and  existing  taxonomies  from  the  non¬ 
military,  civilian  sector.  Candidates  to  start  from  include 

•  ONSET  Knowledges  and  Education  Requirements  taxonomies 

•  OPM’s  Technical  Competencies  taxonomy 

•  Existing  taxonomies  of  educational  and  instructional  programs  (e.g.,  from  U.S. 

Department  of  Education) 

Where  feasible,  this  information  could  be  supplemented  by  a  review  of  college  and  technical 
school  course  catalogs. 

Step  1.2:  Create  an  Excel  Workbook  to  document  the  CK  development  process.  Creation 
of  the  Workbook  involves  importing  candidates  (from  Step  1.1)  determined  to  be  most  useful 
and  coding  the  selected  CK  descriptors  so  that  the  CKs  generated  can  be  tracked/linked  back  to 
the  relevant  source  material.  A  useful  strategy  for  narrowing  down,  or  otherwise  refining,  the 
initial  list  of  candidate  CK  descriptors  is  to  crosswalk  each  set  of  descriptors  against  each  other, 
eliminating  redundancies  as  needed. 

Step  1.3:  Divide  the  work  so  that  each  member  of  the  Descriptor  Development  Team 
focuses  on  and  responsible  for  a  cluster(s)  of  related  CKs  (e.g.,  according  to  major  subject 
matter). 

Step  1.4:  Generate  a  first  draft  of  CKs.  An  important  part  of  this  step  will  involve 
customizing  (some)  preliminary  taxonomy  of  CKs  to  the  Army  by  crosswalking  CKs  with 
Army-specific  information.  To  do  this,  first  refonnat  and  revise  the  candidate  CK  descriptors 
from  the  non-military,  civilian  sector  in  line  with  the  proposed  guidelines  and  specifications,  as 
outlined  here,  to  ensure  consistency  in  breadth,  scope,  and  level  of  generality.  Next,  crosswalk 
this  preliminary  taxonomy  with  the  existing  ASVAB  test  specifications  (e.g.,  Oppler,  Russell, 
Rosse,  Keil,  Meiman,  &  Welsh,  1997).  Finally,  crosswalk  the  preliminary  taxonomy  with  the 
Anny’s  current  Career  Management  Fields  (CFMs).  On  the  basis  of  these  crosswalks,  make 
revisions  and/or  refinements  to  the  preliminary  taxonomy,  as  needed. 

Step  1.5:  Prepare  the  CK  Workbook,  ensuring  that  the  CKs  and  documentation  on  all 
source  materials,  and  so  on,  are  included.  When  finished,  post  the  Workbook  to  the  designated 
folder  on  the  S-drive. 
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Activity  2.  A  Descriptor  Review  Team  reviews  the  preliminary  CK  taxonomy  to  ensure 
consistency  with  proposed  guidelines  and  specifications  and  that  the  statements  provide 
sufficient  and  reasonably  comprehensive  coverage  of  the  domain.  Descriptor  Development  Team 
implements  recommended  edits,  as  needed. 

Activity  3.  Conduct  a  sorting  exercise  with  the  Descriptor  Development  Team  using  the 
preliminary  CK  taxonomy.  The  purpose  of  this  sorting  exercise  is  to  identify  and  recommend 
possible  modifications  and  refinements  to  the  preliminary  CK  taxonomy.  In  brief,  this  sorting 
exercise  will  involve  sorting  the  CKs  into  the  MDs  previously  developed.  A  senior  member  of 
the  Descriptor  Review  Team  will  facilitate  the  sorting  exercise. 

Activity  4.  The  Expert  Advisory  Panel,  high-level  Army  SMEs  (if  available),  and  Army  job 
analysts  review  the  preliminary  CK  taxonomy  and  select  metric(s).  The  purpose  of  this  review  is 
threefold:  (a)  to  identify  and  recommend  modifications  and  edits  to  the  preliminary  CK 
taxonomy;  (b)  to  identify  and  recommend  potential  candidates  for  elimination,  if  needed  (e.g., 
CKs  that  are  not  relevant  to  the  Army);  and  (c)  to  select  metric(s)  for  ratings  jobs  on  their  CKs 
requirements.  The  Descriptor  Development  Team  implements  recommended  edits,  as  needed. 

Activity  5.  Conduct  workshops  with  Army  SMEs  from  the  targeted  jobs.  The  purpose  of 
these  workshops  is  to  collected  feedback  and  preliminary  data  on  the  KWAs  from  Army  SMEs. 

Recommendations  and  Specifications  on  CKs  Metrics 

Accompanying  the  CK  descriptors  are  metrics  (i.e.,  scales)  by  which  Army 
occupations/jobs  could  be  assessed  on  these  same  descriptors.  At  a  minimum,  these  metrics 
should  include  generic,  Likert-type  scales  measuring  the  importance  of  the  CK  to  the 
performance  of  the  occupation/job.  In  particular,  using  the  existing  0*NET  Knowledge  scales 
would  enable  and  facilitate  comparisons  between  the  prototype  job  descriptors  and  existing 
alternatives. 


Work  Interests,  Values,  and  Styles 


Definitions 

Work  interests,  values,  and  styles  are  descriptions  of  enduring  personal  characteristics 
representing  work-related  preferences  a  Soldier  (or  Officer)  needs  to  perform  or  to  persist  in  an 
Army  occupation  or  job,  or  cluster  of  occupations  or  jobs.  Work  interests  describe  a  Soldier’s  (or 
Officer’s)  preferences  for,  or  likes  and  dislikes,  regarding  specific  work-related  activities  (e.g., 
performing  physically-oriented  work  activities  like  running,  jumping,  lifting  heavy  objects,  etc.). 
Work  values  describe  a  Soldier’s  (or  Officer’s)  goals,  beliefs,  or  ideals  regarding  specific  work- 
related  requirements  (e.g.,  working  independently,  performing  a  variety  of  activities,  etc.).  Work 
styles  describe  a  Soldier’s  (or  Officer’s)  behavioral  tendencies,  habits,  or  styles  reflecting 
preferences  for  specific  work-related  requirements  (e.g.,  working  cooperatively  with  others, 
demonstrating  initiative,  etc.). 
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Objectives  and  Purposes  to  be  Served 

Like  CKs,  work  interests,  values,  and  styles  are  intended  to  provide  a  common  basis,  or 
set  of  cross-descriptors,  on  which  Army  jobs  could  be  rated  or  scaled  for  purposes  of  clustering 
jobs  (i.e.,  to  guide  the  sampling  of  Army  jobs  for  criterion-related  validation  studies;  to 
generalize,  or  transport,  criterion-related  validity  estimates  and  predicted  performance  equations 
across  jobs;  to  identify  potential  candidates  for  “mid-range”  criterion  measures,  in  particular 
knowledge  tests). 

Proposed  Steps  and  Procedure  for  Generating  Work  Interests,  Values,  and  Styles  Descriptors 

The  following  are  the  proposed  steps  and  procedure  for  generating  work  interests,  values, 
and  styles  descriptors.  The  output  of  this  process  will  be  a  set  of  work  interests,  values,  and 
styles  for  use  in  job  analysis  workshops  and  questionnaires  with  proponent  SMEs. 

Activity  1.  Starting  with  a  pool  of  candidate  descriptors,  the  Descriptor  Development 
Team  generates  preliminary  taxonomies  of  work  interests,  values,  and  styles.  Completing  this 
activity  involves  the  following  steps 

Step  1.1:  Review  candidate  work  interests,  values,  and  styles  descriptors  and  existing 
taxonomies.  Candidates  to  start  from  include: 

•  The  work  interests,  values,  and  styles  taxonomies  formulated  and  used  as  the  basis  for 
developing  predictor  measures  for  the  Army21  projects  (Perform21,  Select21,  Army 
Class) 

•  0*NET  Work  Interests  and  Work  Styles  taxonomies 

Step  1.2:  Create  an  Excel  Workbook  to  document  the  work  interests,  values,  and  styles 
development  process.  Import  candidate  descriptors  from  Step  1 . 1  and  then  code  the  selected 
work  interests,  values,  and  styles  descriptors  so  that  they  can  be  tracked  or  linked  back  to  the 
relevant  source  material. 

Step  1.3:  Generate  an  initial  draft  of  work  interests,  values,  and  styles.  A  useful  strategy 
for  narrowing  down,  or  otherwise  refining,  the  initial  list  of  candidate  work 
interests/values/styles  descriptors  is  to  crosswalk  each  set  of  descriptors  against  each  other, 
eliminating  redundancies  as  needed. 

Step  1.4:  Prepare  the  Work  Interests/Values/Styles  Workbook,  ensuring  that  the  final 
draft  set  of  descriptors  and  documentation  on  all  source  materials,  and  so  on,  are  included.  When 
finished,  post  the  Workbook  to  the  designated  folder  on  the  S-drive. 

Activity  2.  A  Descriptor  Review  Team  reviews  the  preliminary  work  interests,  values  and 
styles  taxonomies  to  ensure  consistency  with  proposed  guidelines  and  specifications  and  that  the 
statements  provide  sufficient  and  reasonably  comprehensive  coverage  of  the  domain.  Descriptor 
Development  Team  implements  recommended  edits,  as  needed. 
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Activity  3.  Conduct  a  sorting  exercise  with  the  Descriptor  Development  Team  using  the 
preliminary  work  interests,  values,  and  styles  taxonomies .  The  purpose  of  this  sorting  exercise  is 
to  identify  and  recommend  possible  modifications  and  refinements  to  the  preliminary  work 
interests,  values,  and  styles  taxonomies.  In  brief,  this  sorting  exercise  will  involve  sorting  the 
work  interests,  values,  and  styles  into  the  MDs  previously  developed.  A  senior  member  of  the 
Descriptor  Review  Team  will  facilitate  the  sorting  exercise. 

Activity  4.  The  Expert  Advisory  Panel,  high-level  Army  SMEs  (if  available),  and  Army  job 
analysts  review  the  preliminary  work  interests,  values,  and  styles  taxonomies  and  select 
metric(s).  The  purpose  of  this  review  is  threefold:  (a)  to  identify  and  recommend  modifications 
and  edits  to  the  preliminary  work  interests,  values,  and  styles  taxonomies;  (b)  to  identify  and 
recommend  potential  candidates  for  elimination,  if  needed  (e.g.,  work  interests/values/styles  that 
are  not  relevant  to  the  Anny);  and  (c)  to  select  metric(s)  for  ratings  jobs  on  the  chosen 
descriptors.  The  Descriptor  Development  Team  implements  recommended  edits,  as  needed. 

Activity  5.  Conduct  workshops  with  Army  SMEs  from  the  targeted  jobs.  The  purpose  of 
these  workshops  is  to  collected  feedback  and  preliminary  data  on  the  work  interests,  values,  and 
styles  from  Army  SMEs. 

Recommendations  and  Specifications  on  Work  Interests/Values/Styles  Metrics 

Accompanying  the  work  interests/values/styles  descriptors  are  metrics  (i.e.,  scales)  on 
which  Army  occupations  or  jobs  could  be  rated  or  scaled.  At  a  minimum,  these  metrics  should 
include  Likert-type  scales  measuring  the  importance  of  the  work  interest/value/style  to  the  Anny 
occupation/job. 


Abilities  and  Aptitudes 


Definitions 

Abilities  and  aptitudes  are  descriptions  of  enduring  personal  characteristics  that  enable  a 
Soldier  (or  Officer)  to  perform  the  mental,  spatial,  physical,  and  physiological  requirements 
underlying  the  core  duties  and  work  activities,  technical  and  non-technical,  required  of  an  Army 
occupation  or  job,  or  cluster  of  occupations  or  jobs. 

Objectives  and  Purposes  to  be  Served 

Like  CKs  and  other  work  interests/values/styles,  abilities  and  aptitudes  are  intended  to 
provide  a  common  basis,  or  set  of  cross-descriptors,  on  which  Army  jobs  could  be  rated  or  scaled 
for  purposes  of  clustering  jobs  (i.e.,  to  guide  the  sampling  of  Army  jobs  for  criterion-related 
validation  studies;  to  generalize,  or  transport,  criterion-related  validity  estimates  and  predicted 
performance  equations  across  jobs;  to  identify  potential  candidates  for  “mid-range”  criterion 
measures,  in  particular  knowledge  tests). 
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Proposed  Steps  and  Procedure  for  Generating  Abilities  and  Aptitudes  Descriptors 

The  following  are  the  proposed  steps  and  procedure  for  generating  abilities  and  aptitudes 
descriptors.  The  output  of  this  process  will  be  a  set  of  abilities  and  aptitudes  descriptors  for  use 
in  job  analysis  workshops  and  questionnaires  with  proponent  SMEs. 

Activity  1.  Starting  with  a  pool  of  candidate  descriptors,  the  Descriptor  Development 
Team  generates  a  preliminary  taxonomy  of  abilities  and  aptitudes.  Completing  this  activity 
involves  the  following  steps 

Step  1.1:  Review  the  0*NET  Ability  taxonomy.  The  0*NET  Ability  taxonomy 
represents  a  comprehensive  and  well-researched  taxonomy  of  abilities  and  aptitudes  required  of 
most,  if  not  all,  occupations  or  jobs. 

Step  1.2:  Create  an  Excel  Workbook  to  document  the  abilities  and  aptitudes  taxonomy 
development  process.  Import  0*NET  Ability  descriptors  and  then  code  them  so  that  they  can  be 
tracked  or  linked  back  to  the  relevant  source  material,  as  needed. 

Step  1.3:  Refine  or  modify  the  0*NET  Ability  taxonomy,  as  needed.  Cross-walk  the 
0*NET  Ability  taxonomy  against  more  recent  ability  and  aptitude  taxonomies,  eliminating 
redundancies  as  needed. 

Step  1.4:  Prepare  the  Abilities  and  Aptitudes  Workbook,  ensuring  that  the  final  draft  set 
of  descriptors  and  documentation  on  all  source  materials,  and  so  on,  are  included.  When  finished, 
post  the  Workbook  to  the  designated  folder  on  the  S-drive. 

Activity  2.  A  Descriptor  Review  Team  reviews  the  preliminary  abilities  and  aptitudes 
taxonomy  to  ensure  consistency  with  proposed  guidelines  and  specifications  and  that  the 
statements  provide  sufficient  and  reasonably  comprehensive  coverage  of  the  domain.  Descriptor 
Development  Team  implements  recommended  edits,  as  needed. 

Activity  3.  Conduct  a  sorting  exercise  with  the  Descriptor  Development  Team  using  the 
preliminary  abilities  and  aptitudes  taxonomy.  The  purpose  of  this  sorting  exercise  is  to  identify 
and  recommend  possible  modifications  and  refinements  to  the  preliminary  abilities  and  aptitudes 
taxonomy.  In  brief,  this  sorting  exercise  will  involve  mapping  the  abilities  and  aptitudes 
descriptors  to  the  MDs  previously  developed.  A  senior  member  of  the  Descriptor  Review  Team 
will  facilitate  the  sorting  exercise. 

Activity  4.  The  Expert  Advisory  Panel,  high-level  Army  SMEs  (if  available),  and  Army  job 
analysts  review  the  preliminary  abilities  and  aptitudes  taxonomies  and  select  rnetric(s).  The 
purpose  of  this  review  is  threefold:  (a)  to  identify  and  recommend  modifications  and  edits  to  the 
preliminary  abilities  and  aptitudes  taxonomy;  (b)  to  identify  and  recommend  potential  candidates 
for  elimination,  if  needed  (e.g.,  abilities  and  aptitudes  that  are  not  relevant  to  the  Anny);  and  (c) 
to  select  metric(s)  for  ratings  jobs  on  the  final  set  of  abilities  and  aptitudes  descriptors.  The 
Descriptor  Development  Team  implements  recommended  edits,  as  needed. 
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Activity  5.  Conduct  workshops  with  Army  SMEs  from  the  targeted  jobs.  The  purpose  of 
these  workshops  is  to  collected  feedback  and  preliminary  data  on  the  abilities  and  aptitudes  from 
Army  SMEs. 

Recommendations  and  Specifications  on  Abilities  and  Aptitudes  Metrics 

Accompanying  the  abilities  and  aptitudes  descriptors  are  metrics  (i.e.,  scales)  on  which 
Army  occupations  or  jobs  could  be  rated  or  scaled.  At  a  minimum,  these  metrics  should  include 
Likert-type  scales  measuring  the  importance  of  the  ability  or  aptitude  to  the  Army 
occupation/job. 
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Appendix  B 


Description  and  Layout  of  the  Relational  Database 
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Overview  of  the  Database  Design 


The  database  developed  for  this  project  was  designed  as  a  relational  database,  consisting 
of  a  series  of  reference  and  data  tables  linked  by  unique  identifiers  associated  with  an  Army  job 
and  the  descriptor  domain  being  sampled.9  The  database’s  primary  objective,  near  term,  is  to 
systematically  store  ratings  data  collected  using  the  prototype  approach  across  five  domains  -  (a) 
Work  Activities,  (b)  Occupation  Specific  Knowledge  and  Skills,  (c)  Work  Context,  (d)  Work 
Interests,  Values  and  Styles,  and  (e)  Aptitudes  and  Abilities.  Over  time,  a  user  interface  could  be 
developed  that  allows  for  easy  reporting  and  editing  capabilities. 

Reference  Tables 

The  following  tables  contain  reference  data  that  will  be  used  across  domain. 

*  indicates  a  field  used  in  the  Primary  Key. 

**  indicates  a  field  used  in  a  secondary  index. 


JOB  REF 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID** 

Integer 

2 

Unique  auto  generated  ID  for  each  job. 

MOS  ID* 

Varchar 

3 

From  MOS  REF 

SKILL  LVL  ID* 

Tinyint 

1 

From  SKILL  LVL  REF 

DUTY  POS  ID* 

Varchar 

5 

From  DUTY  POS  REF 

ASSIGNMENT  ID* 

Varchar 

5 

From  ASSIGNMENT  REF 

MOS  REF 


Field  Name 

Type 

Length 

Field  Description 

MOS  ID* 

Varchar 

7 

Unique  ID  for  each  MOS 

MOS  TITLE 

Varchar 

80 

Title  of  MOS 

CMF  ID 

Varchar 

2 

From  CMF  REF 

APT  CLUSTER  ID 

Varchar 

2 

From  APT  CLUSTER  REF 

CMF  REF 


Field  Name 

Type 

Length 

Field  Description 

CMF  ID* 

Varchar 

6 

Unique  ID  for  each  CMF 

CMF  TITLE 

Varchar 

80 

Title  of  CMF 

APT  CLUSTER  REF 


Field  Name 

Type 

Length 

Field  Description 

APT  CLUSTER  ID* 

Varchar 

6 

Unique  ID  for  each  Aptitude  Area  Cluster 

APT  TITLE 

Varchar 

80 

Title  of  Aptitude  Area  Cluster 

9  A  “job”  can  be  defined  by  any  combination  of  the  following:  (a)  MOS,  (b)  Skill  Level,  (c)  Duty  Position,  or  (d) 
Assignment. 
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SKILL  LYL  REF 


Field  Name 

Type 

Length 

Field  Description 

SKILL  LVL  ID* 

Tinyint 

12 

Unique  ID  for  each  Skill  Level  (1-5) 

SKILL  LVL  TITLE 

Varchar 

80 

Skill  Level  Title 

DUTY  POS  REF 


Field  Name 

Type 

Length 

Field  Description 

DUTY  POS  ID* 

Varchar 

5 

Unique  ID  for  each  Duty  Position 

DUTY  POS  TITLE 

Varchar 

80 

Duty  Position  Title 

ASSIGNMENT  REF 


Field  Name 

Type 

Length 

Field  Description 

ASSIGNMENT  ID* 

Varchar 

5 

Unique  ID  for  each  Assignment 

ASSIGNMENT  TITLE 

Varchar 

80 

Assignment  Title 

METRIC  REF 


Field  Name 

Type 

Length 

Field  Description 

METRIC  DOM  ID* 

Small  Int 

2 

Unique  ID  for  each  Metric 
Category/Value/Domain 

METRIC  VALUE* 

Varchar 

1 

Value  for  a  particular  Metric  (i.e.  1-7) 

METRIC  DESC 

Varchar 

20 

Description  of  Metric  Value  (i.e.  Very 
Important) 

METRIC  x  DOMAIN 


Field  Name 

Type 

Length 

Field  Description 

METRIC  DOM  ID** 

Small  Int 

2 

Unique  ID  for  each  Metric  Category  x 
Domain 

METRIC  CAT  ID* 

Small  Int 

2 

Unique  ID  for  Each  Metric  Category 

DOMAIN  ID* 

Varchar 

4 

Domain  ID  from  DOMAIN  REF  table. 

DOMAIN  REF 


Field  Name 

Type 

Length 

Field  Description 

DOMAIN  ID* 

Varchar 

4 

Unique  ID  for  each  Domain 

DOMAIN  TITLE 

Varchar 

50 

Domain  Title 

METRIC  CAT  REF 


Field  Name 

Type 

Length 

Field  Description 

METRIC  CAT  ID* 

Small  Int 

2 

Unique  ID  for  Each  Metric  Category 
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METRIC  CATEGORY _ Varchar  12  Metric  Category  (Frequency,  Importance) 


RATER  REF 


Field  Name 

Type 

Length 

Field  Description 

RATER  ID* 

Tiny  Int 

1 

Unique  ID  for  Each  Rater  Type. 

RATER  DESC 

Varchar 

60 

Rater  Type  (Overall,  Supervisor,  Incumbent) 

Work  Activities 

The  following  tables  describe  the  way  work  activity  data  will  be  stored. 


MD  REF 


Field  Name 

Type 

Length 

Field  Description 

MD  ID* 

Varchar 

6 

Unique  identifier  for  Major  Duty 

MD  TITLE 

Varchar 

80 

MD  title 

MD  DESCRIPTION 

Varchar 

255 

MD  Description 

MD  CAT  ID 

Varchar 

9 

From  MD  CAT  REF 

MD  CAT  REF 


Field  Name 

Type 

Length 

Field  Description 

MD  CAT  ID* 

Varchar 

9 

Unique  identifier  for  Major  Duty  Category 

MD  CAT  TITLE 

Varchar 

50 

Job-specific  tech,  non-job  specific  tech, 
non-tech 

KWAC  REF 


Field  Name 

Type 

Length 

Field  Description 

KWAC  ID* 

Varchar 

8 

Unique  identifier  for  Key  Work  Activity  Category 

KWAC  DESC 

Varchar 

200 

Description  of  KWAC 

MD  x  KWAC 


Field  Name 

Type 

Length 

Field  Description 

MD  ID* 

Varchar 

6 

Unique  identifier,  From  MD  REF 

KWAC  ID* 

Varchar 

8 

Unique  identifier,  From  KWAC  REF 

KWAS  REF 


Field  Name 

Type 

Length 

Field  Description 

KWAS  ID* 

Varchar 

10 

Unique  identifier  for  Key  Work  Activity  Statement 

KWAS  DESC 

Varchar 

150 

Description  of  KWAS 
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SOURCE  CD 

Varchar 

50 

Source  Code 

SOURCE  STMT 

Varchar 

150 

Source  Statement 

SOURCE  STMT  DT 

Date 

8 

Effective  date  of  source  statement 

KWAC  ID 

Varchar 

8 

From  KWAC  ID;  Key  Work  Activity  Code  KWAS 
is  associated  with 

JOB  x  MD 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID* 

Integer 

2 

From  JOB  REF 

MD  ID* 

Varchar 

6 

From  MD  REF 

METRIC  DOM  ID* 

Small  Int 

2 

From  METRIC  REF;  Metric  Category 

METRIC  SIZE 

Small  Int 

2 

Sample  Size  for  this  metric. 

METRIC  MEAN 

Double 

8 

Mean  for  this  metric. 

METRIC  ICC1 

Double 

8 

Intraclass  Correlation  1 

METRIC  ICC2 

Double 

8 

Intraclass  Correlation  2 

METRIC  STD 

Double 

8 

Standard  Deviation  for  this  metric 

RATER  ID* 

Tiny  Int 

1 

From  RATER  REF  table 

METRIC  STAT1 

Double 

8 

Place  Holder 

METRIC  STAT2 

Double 

8 

Place  Holder 

METRIC  DT* 

Date 

8 

Date  metric  calculated 

METRIC  MEDIAN 

Double 

8 

Median  for  this  Metric 

METRIC  SKEWNESS 

Double 

8 

Skewness  for  this  Metric 

JOB  x  KWAC 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID* 

Integer 

2 

From  JOB  REF 

KWAC  ID* 

Varchar 

8 

From  KWAC  REF 

METRIC  DOM  ID* 

Small  Int 

2 

From  METRIC  REF;  Metric  Category 

METRIC  SIZE 

Small  Int 

2 

Size  for  this  metric. 

METRIC  MEAN 

Double 

8 

Mean  for  this  metric. 

METRIC  ICC1 

Double 

8 

Intraclass  Correlation  1 

METRIC  ICC2 

Double 

8 

Intraclass  Correlation  2 

METRIC  STD 

Double 

8 

Standard  Deviation  for  this  metric 

RATER  ID* 

Tiny  Int 

1 

From  RATER  REF  table 

METRIC  STAT1 

Double 

8 

Place  Holder 

METRIC  STAT2 

Double 

8 

Place  Holder 

METRIC  DT* 

Date 

8 

Date  metric  calculated 

METRIC  MEDIAN 

Double 

8 

Median  for  this  Metric 

METRIC  SKEWNESS 

Double 

8 

Skewness  for  this  Metric 
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JOB  x  KWAS 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID* 

Integer 

2 

From  JOB  REF 

KWAS  ID* 

Varchar 

10 

From  KWAS  REF 

METRIC  DOM  ID* 

Small  Int 

2 

From  METRIC  REF;  Metric  Category 

METRIC  SIZE 

Small  Int 

2 

Size  for  this  metric. 

METRIC  MEAN 

Double 

8 

Mean  for  this  metric. 

METRIC  ICC1 

Double 

8 

Intraclass  Correlation  1 

METRIC  ICC2 

Double 

8 

Intraclass  Correlation  2 

METRIC  STD 

Double 

8 

Standard  Deviation  for  this  metric 

RATER  ID* 

Tiny  Int 

1 

From  RATER  REF  table 

METRIC  STAT1 

Double 

8 

Place  Holder 

METRIC  STAT2 

Double 

8 

Place  Holder 

METRIC  DT* 

Date 

8 

Date  metric  calculated 

METRIC  MEDIAN 

Double 

8 

Median  for  this  Metric 

METRIC  SKEWNESS 

Double 

8 

Skewness  for  this  Metric 

WET  REF 


Field  Name 

Type 

Length 

Field  Description 

WET  ID* 

Varchar 

4 

Unique  ID  for  WET,  this  is  the  FSC  Code. 

WET  DESC 

Varchar 

100 

Description  of  WET 

JOB  x  WET 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID* 

Integer 

2 

From  JOB  REF 

KWAS  ID* 

Varchar 

10 

From  KWAS  REF 

WET  ID* 

Varchar 

4 

From  WET  REF 

METRIC  DOM  ID* 

Small  Int 

2 

From  METRIC  REF;  Metric  Category 

METRIC  SIZE 

Small  Int 

2 

Size  for  this  metric. 

METRIC  MEAN 

Double 

8 

Mean  for  this  metric. 

METRIC  ICC1 

Double 

8 

Intraclass  Correlation  1 

METRIC  ICC2 

Double 

8 

Intraclass  Correlation  2 

METRIC  STD 

Double 

8 

Standard  Deviation  for  this  metric 

RATER  ID* 

Tiny  Int 

1 

From  RATER  REF  table 

METRIC  STAT1 

Double 

8 

Place  Holder 

METRIC  STAT2 

Double 

8 

Place  Holder 

METRIC  DT* 

Date 

8 

Date  metric  calculated 

METRIC  MEDIAN 

Double 

8 

Median  for  this  Metric 

METRIC  SKEWNESS 

Double 

8 

Skewness  for  this  Metric 
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Occupation  Specific  Knowledges  and  Skills 


The  following  tables  describe  the  data  to  be  collected  and  stored  under  the  Occupation 
Specific  Knowledges  and  Skills  domain. 


CK  REF 


Field  Name 

Type 

Length 

Field  Description 

CK  ID* 

Varchar 

8 

Unique  identifier  for  Content  Knowledges  (CK) 

CK  TITLE 

Varchar 

80 

Title  of  CK 

CK  DESC 

Varchar 

255 

Description  of  CK 

CK  CAT  ID 

Varchar 

2 

From  CK  CAT  REF 

CK  CAT  REF 


Field  Name 

Type 

Length 

Field  Description 

CK  CAT  ID* 

Varchar 

2 

Unique  identifier  for  CK  Categories. 

CK  CAT  TITLE 

Varchar 

50 

CK  Category  Titles 

JOB  x  CK 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID* 

Integer 

2 

From  JOB  REF 

CK  ID* 

Varchar 

8 

From  CK  REF 

METRIC  DOM  ID* 

Small  Int 

2 

From  METRIC  REF;  Metric  Category 

METRIC  SIZE 

Small  Int 

2 

Size  for  this  metric. 

METRIC  MEAN 

Double 

8 

Mean  for  this  metric. 

METRIC  ICC1 

Double 

8 

Intraclass  Correlation  1 

METRIC  ICC2 

Double 

8 

Intraclass  Correlation  2 

METRIC  STD 

Double 

8 

Standard  Deviation  for  this  metric 

RATER  ID* 

Tiny  Int 

1 

From  RATER  REF  table 

METRIC  STAT1 

Double 

8 

Place  Holder 

METRIC  STAT2 

Double 

8 

Place  Holder 

METRIC  DT* 

Date 

8 

Date  metric  calculated 

METRIC  MEDIAN 

Double 

8 

Median  for  this  Metric 

METRIC  SKEWNESS 

Double 

8 

Skewness  for  this  Metric 
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Work  Context 


The  following  tables  describe  the  data  to  be  collected  and  stored  under  the  Work  Context 
domain. 


WORK  CONTEXT  REF 


Field  Name 

Type 

Length 

Field  Description 

WORK  CONT  ID* 

Varchar 

8 

Unique  identifier  for  Work  Context 

WORK  CONT  TITLE 

Varchar 

80 

Title  of  Work  Context 

WORK  CONT  DESC 

Varchar 

255 

Description  of  Work  Context 

WORK  CONT  CAT  ID 

Varchar 

2 

From  WORK  CONTEXT  CAT  REF 

WORK  CONTEXT  CAT  REF 


Field  Name 

Type 

Length 

Field  Description 

WORK  CONT  CAT  ID* 

Varchar 

9 

Unique  Identifier  for  Work  Context 
Categories 

WORK  CONT  CAT  TITLE 

Varchar 

50 

Work  Context  Category  Title. 

JOB  x  WORK  CONTEXT 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID* 

Integer 

2 

From  JOB  REF 

WORK  CONT  ID* 

Varchar 

8 

From  WORK  CONTEXT  REF 

METRIC  DOM  ID* 

Small  Int 

2 

From  METRIC  REF;  Metric  Category 

METRIC  SIZE 

Small  Int 

2 

Size  for  this  metric. 

METRIC  MEAN 

Double 

8 

Mean  for  this  metric. 

METRIC  ICC1 

Double 

8 

Intraclass  Correlation  1 

METRIC  ICC2 

Double 

8 

Intraclass  Correlation  2 

METRIC  STD 

Double 

8 

Standard  Deviation  for  this  metric 

RATER  ID* 

Tiny  Int 

1 

From  RATER  REF  table 

METRIC  STAT1 

Double 

8 

Place  Holder 

METRIC  STAT2 

Double 

8 

Place  Holder 

METRIC  DT* 

Date 

8 

Date  metric  calculated 

METRIC  MEDIAN 

Double 

8 

Median  for  this  Metric 

METRIC  SKEWNESS 

Double 

8 

Skewness  for  this  Metric 

B-9 


Work  Interests,  Values,  and  Styles 


The  following  tables  describe  the  data  to  be  collected  and  stored  under  the  Work 
Interests,  Values,  and  Styles  domain. 


INT  REF 


Field  Name 

Type 

Length 

Field  Description 

INT  ID* 

Varchar 

8 

Unique  identifier  for  Interest/Values/Style 

INT  TITLE 

Varchar 

80 

T itle  of  Interest/V alues/Style 

INT  DESC 

Varchar 

255 

Description  of  Interest/Values/Style 

INT  CAT  ID 

Varchar 

2 

From  INT  CAT  REF 

INT  CAT  REF 


Field  Name 

Type 

Length 

Field  Description 

INT  CAT  ID* 

Varchar 

2 

Unique  ID  for  Interest/V  alues/Style  Category 

INT  CAT  TITLE 

Varchar 

50 

Interest/Values/Style  Category  Title 

JOB  x  INT 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID* 

Integer 

2 

From  JOB  REF 

INT  ID* 

Varchar 

8 

From  INT  REF 

METRIC  DOM  ID* 

Small  Int 

2 

From  METRIC  REF;  Metric  Category 

METRIC  SIZE 

Small  Int 

2 

Size  for  this  metric. 

METRIC  MEAN 

Double 

8 

Mean  for  this  metric. 

METRIC  ICC1 

Double 

8 

Intraclass  Correlation  1 

METRIC  ICC2 

Double 

8 

Intraclass  Correlation  2 

METRIC  STD 

Double 

8 

Standard  Deviation  for  this  metric 

RATER  ID* 

Tiny  Int 

1 

From  RATER  REF  table 

METRIC  STAT1 

Double 

8 

Place  Holder 

METRIC  STAT2 

Double 

8 

Place  Holder 

METRIC  DT* 

Date 

8 

Date  metric  calculated 

METRIC  MEDIAN 

Double 

8 

Median  for  this  Metric 

METRIC  SKEWNESS 

Double 

8 

Skewness  for  this  Metric 
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Abilities  and  Aptitudes 


The  following  tables  describe  the  data  to  be  collected  and  stored  under  the  Abilities  and 
Aptitudes  domain. 


APT  REF 


Field  Name 

Type 

Length 

Field  Description 

APT  ID* 

Varchar 

8 

Unique  identifier  for  Aptitudes  and  Abilities 

APT  TITLE 

Varchar 

80 

Title  for  Aptitudes  and  Abilities 

APT  DESC 

Varchar 

255 

Description  for  Aptitudes  and  Abilities 

APT  CAT  ID 

Varchar 

2 

From  APT  CAT  REF 

APT  CAT  REF 


Field  Name 

Type 

Length 

Field  Description 

APT  CAT  ID* 

Varchar 

6 

Unique  ID  for  Aptitudes  and  Ability  Category. 

APT  CAT  TITLE 

Varchar 

50 

Aptitudes  and  Ability  Category  Title. 

JOB  x  APT 


Field  Name 

Type 

Length 

Field  Description 

JOB  ID* 

Integer 

2 

From  JOB  REF 

APT  ID* 

Varchar 

8 

From  APT  REF 

METRIC  DOM  ID* 

Small  Int 

2 

From  METRIC  REF;  Metric  Category 

METRIC  SIZE 

Small  Int 

2 

Size  for  this  metric. 

METRIC  MEAN 

Double 

8 

Mean  for  this  metric. 

METRIC  ICC1 

Double 

8 

Intraclass  Correlation  1 

METRIC  ICC2 

Double 

8 

Intraclass  Correlation  2 

METRIC  STD 

Double 

8 

Standard  Deviation  for  this  metric 

RATER  ID* 

Tiny  Int 

1 

From  RATER  REF  table 

METRIC  STAT1 

Double 

8 

Place  Holder 

METRIC  STAT2 

Double 

8 

Place  Holder 

METRIC  DT* 

Date 

8 

Date  metric  calculated 

METRIC  MEDIAN 

Double 

8 

Median  for  this  Metric 

METRIC  SKEWNESS 

Double 

8 

Skewness  for  this  Metric 
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Appendix  C 


Work  Activities  Questionnaires 
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Army  Work  Activities  Questionnaire 

11B  Infantryman 
Instructions 


The  purpose  of  this  questionnaire  is  to  find  out  about  the  work  activities  performed  by  first-term 
Soldiers  in  your  MOS.  As  you  complete  this  questionnaire,  consider  the  typical  first-term  Soldier 
in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a 
typical  unit  assignment  and  position  and  at  least  one  series  of  Green- Amber-Red  training  cycles 
and  unit  operational  requirements. 

This  questionnaire  consists  of  a  series  of  work  activities  describing  what  first-term  Soldiers  in 
your  MOS  do,  to  whom  or  what  it  is  done,  and  potentially  for  what  purpose  it  is  done.  For 
example: 


Operate  radio  communications  equipment. 


For  each  work  activity,  you  will  be  asked  to  make  three  ratings. 

The  first  involves  rating  each  work  activity  in  terms  of  its  importance  to  the  job  of first-term 
Soldiers  in  your  MOS.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Important 

Somewhat 

Important 

Important 

Very  Important 

Extremely 

Important 

1 

2 

3 

4 

5 

The  second  involves  rating  each  work  activity  in  terms  of  how  difficult  it  is  for  first  term  Soldiers  in 
your  MOS  to  learn  to  perform  the  activity  to  standard  through  classroom  training.  You  will  use  the 
following  scale  to  make  this  rating: 


Very  Easy  to 
Learn 

Somewhat  Easy 
to  Learn 

Neither  Easy 
nor  Difficult  to 
Learn 

Somewhat 
Difficult  to 
Learn 

Very  Difficult  to 
Learn 

1 

2 

3 

4 

5 
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The  third  involves  rating  each  work  activity  in  terms  of  how  serious  the  negative  consequences 
would  be  for  the  Soldier ’s  unit  or  the  unit ’s  mission  if  a  first-term  Soldier  in  your  MOS  failed  to 
successfully  perform  the  activity.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Serious 

Somewhat 

Serious 

Serious 

Very  Serious 

Extremely 

Serious 

1 

2 

3 

4 

5 

You  will  complete  this  set  of  ratings  twice.  You  will  first  rate  all  of  the  individual  work 
activities.  You  will  then  rate  the  higher-level  categories  these  work  activities  fall  under. 
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Part  1  -  Individual  Work  Activities 


No. 

Work  Activity 

Importance 

G-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Operate  and  maintain  firearms  and  personal  weapons  (e.g., 
grenade  launchers,  grenades). 

1 

Select  and  prepare  firing  positions  for  firearms  and  personal 
weapons  (e.g.,  grenade  launchers,  grenades). 

2 

Mount  and  dismount  sights,  lights,  and  other  accessories  on  firearms 
and  personal  weapons  (e.g.,  grenade  launchers,  grenades). 

3 

Mount  and  dismount  machine  guns  or  other  personal  weapons  on 
tripods. 

4 

Construct  field-expedient  firing  aids  for  firearms  and  personal  weapons. 

5 

Load,  unload,  and  correct  malfunctions  in  firearms  and  personal 
weapons. 

6 

Aim,  track,  and  fire  firearms  or  personal  weapons  at  targets, 
stationary  or  moving. 

7 

Coordinate  the  firing  of  firearms  or  personal  weapons  with  the 
actions  or  movement  of  others. 

8 

Perform  routine  maintenance  checks  and  services  on  firearms  and 
personal  weapons. 

9 

Troubleshoot  firearm  and  personal  weapon  malfunctions  and 
misfires. 

B 

Operate  and  maintain  armored  vehicle-mounted  weapons  (e.g., 
machine  guns,  rocket  launchers). 

10 

Mount  and  dismount  weapon  (e.g.,  machine  gun,  rocket  launcher)  on 
armored  vehicle. 

11 

Load,  unload,  and  correct  malfunctions  in  armored  vehicle-mounted 
weapon  (e.g.,  machine  gun,  rocket  launcher). 

12 

Aim,  track,  and  fire  armored  vehicle-mounted  weapon  at  targets, 
stationary  or  moving. 

13 

Coordinate  the  firing  of  armored  vehicle-mounted  weapon  with  the 
actions  or  movement  of  others. 

14 

Perform  routine  maintenance  checks  and  services  on  vehicle- 
mounted  weapon. 

15 

Troubleshoot  vehicle-mounted  weapon  malfunctions  and  misfires. 

C 

Engage  in  close  combat  or  self-defense. 

17 

Engage  enemy  in  hand. 

18 

Engage  or  defend  self  against  enemy  with  bayonet  or  other  hand. 

D 

Navigate  and  move  in  the  field. 

19 

Determine  the  direction  or  location  of  self  and  others  (friendly  or 
foe),  with  or  without  the  aid  of  maps  or  other  navigational 
equipment  (e.g.,  compass,  GPS). 

20 

Navigate  from  point  to  point  using  maps  and  navigational  equipment 
(e.g.,  compass,  GPS),  day  or  night. 

21 

Camouflage  or  hide  self  and  equipment  to  avoid  detection. 

22 

Practice  noise,  light,  and  litter  discipline  to  conceal  movement  or 
location. 

23 

Move  self,  weapons,  gear,  or  equipment  through  the  field  and 
around  obstacles  (natural  or  man-made),  under  fire,  day  or  night. 
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No. 

Work  Activity 

Importance 

G-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

24 

Locate,  recover,  and  neutralize  antipersonnel  mines  or  other 
unexploded  ordinance. 

25 

React  to  counterattack,  ambush,  or  other  defensive  combat  situations 
during  movement. 

E 

Drive  and  maintain  armored  wheeled  and  tracked  vehicles. 

26 

Prepare  armored  wheeled  and  tracked  vehicles  for  mission. 

27 

Drive  armored  wheeled  vehicles  alone  or  in  a  convoy  on  a  tactical  or 
non-tactical  mission. 

28 

Drive  armored  tracked  vehicle  alone  or  in  a  convoy  on  a  tactical  or 
non-tactical  mission. 

29 

Perform  routine  maintenance  checks  and  services  on  armored 
wheeled  or  tracked  vehicle. 

F 

Recover  and  tow  armored  wheeled  and  tracked  vehicles. 

30 

Recover  disabled  armored  vehicle  (wheeled  or  tracked)  in  the  field 
using  available  resources  and  equipment. 

31 

Prepare  and  tow  disabled  armored  vehicle  (wheeled  or  tracked) 
using  standard  equipment  and  vehicles. 

G 

Set-up,  operate,  and  maintain  secure  communications  equipment. 

32 

Set-up  and  install  secure  ground  and  mobile  communications  equipment 
(e.g.,  ground  and  airborne  radios,  tactical  telephones). 

33 

Operate  ground  and  mobile  communications  equipment  (e.g.,  ground 
and  airborne  radios,  tactical  telephones). 

34 

Perform  routine  maintenance  checks  and  services  on  mobile 
communications  equipment  (e.g.,  ground  and  airborne  radios, 
tactical  telephones). 

35 

Troubleshoot  mobile  communications  equipment  (e.g.,  ground  and 
airborne  radios,  tactical  telephones). 

36 

Identify  and  respond  to  an  electronic  attack  and  other  security  threats 
to  a  secure  communications  network. 

37 

Encode  and  decode  messages  delivered  over  secure  ground  and 
mobile  communications  equipment  (e.g.,  ground  and  airborne 
radios,  tactical  telephones). 

H 

Protect  against  chemical,  biological,  radiological  and  nuclear 
(CBRN)  attacks  or  hazards. 

38 

Prepare  for  or  react  to  a  CBRN  attack  or  hazard  by  protecting  self 
and  others  using  appropriate  gear  or  equipment. 

39 

Detect,  monitor,  or  measure  CBRN  agents  or  levels. 

40 

Decontaminate  self  or  equipment  using  decontamination  kits. 

I 

Perform  first  aid  and  treat  casualties. 

41 

Evaluate  a  casualty's  vital  signs,  status,  and  severity  of  their  injuries 
(e.g.,  blood  loss,  shock). 

42 

Perform  CPR,  prevent  shock,  clear  the  throat  of  a  casualty,  or 
perform  other  basic  first  aid. 

43 

Perform  first  aid  for  an  open  wound  or  other  injuries  (e.g.,  bums, 
bone  fractures)  received  during  combat. 

44 

Perform  first  aid  for  an  open  wound  or  other  injuries  (e.g.,  bums, 
bone  fractures)  received  in  a  non-combat  situation. 

45 

Perform  first  aid  for  a  CBRN  casualty. 
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No. 

Work  Activity 

Importance 

G-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

J 

Evacuate  casualties  or  remains. 

46 

Administer  an  IV  and  stabilize  casualty  for  transport. 

47 

Transport  a  casualty,  on  foot  or  using  a  vehicle  or  other  transport 
equipment. 

48 

Request  a  medical  evacuation. 

49 

Recover,  evacuate,  or  inter  remains. 

K 

Set-up,  operate,  and  maintain  secure  communications  equipment. 

50 

Set-up  and  install  secure  ground  and  mobile  communications  equipment 
(e.g.,  ground  and  airborne  radios,  tactical  telephones). 

51 

Operate  ground  and  mobile  communications  equipment  (e.g.,  ground 
and  airborne  radios,  tactical  telephones). 

52 

Perform  routine  maintenance  checks  and  services  on  mobile 
communications  equipment  (e.g.,  ground  and  airborne  radios, 
tactical  telephones). 

53 

Troubleshoot  mobile  communications  equipment  (e.g.,  ground  and 
airborne  radios,  tactical  telephones). 

54 

Identify  and  respond  to  an  electronic  attack  and  other  security  threats 
to  a  secure  communications  network. 

55 

Encode  and  decode  messages  delivered  over  secure  ground  and 
mobile  communications  equipment  (e.g.,  ground  and  airborne 
radios,  tactical  telephones). 

L 

Guard,  search,  and  monitor  detainees  or  prisoners. 

56 

Guard  or  protect  detainees  or  prisoners. 

57 

Search  detainees  or  prisoners  for  weapons  or  prohibited  items. 

58 

Restrain  or  subdue  detainees  or  prisoners  using  the  appropriate  level 
of  force. 

59 

Monitor  detainee  or  prisoner  activity. 

60 

Escort  detainees  or  prisoners  to  a  new  location,  inside  or  outside  a 
detainment  or  corrections  facility. 

M 

Guard,  search,  or  patrol  areas  or  locations. 

61 

Guard  or  patrol  an  assigned  area(s)  or  location)  s). 

62 

Search  an  area  or  location  for  a  person(s)  or  object)  s). 

63 

Obtain  information  from  the  general  public  or  potential  witnesses  on 
suspicious  activity  in  area  or  location. 

64 

Apprehend,  search,  and  detain  a  person) s)  for  suspicious  activity. 

65 

Restrain  or  subdue  a  person) s)  using  the  appropriate  level  of  force. 

66 

Defuse  a  situation) s)  using  interpersonal  skills. 

N 

Guard  and  control  vehicle  and  pedestrian  traffic. 

67 

Guard  or  patrol  key  access  points  or  entrances  to  restricted  areas 
(e.g.,  dismount  points,  roadblocks,  or  checkpoints). 

68 

Search  or  inspect  vehicles  or  persons  for  explosive  devices  or 
prohibited  items. 

69 

Direct  or  control  vehicle  or  pedestrian  traffic. 

70 

Monitor  vehicle  or  pedestrian  traffic  for  suspicious  activity. 

71 

Direct  or  control  the  behavior  of  groups  or  crowds  of  non- 
combatants,  under  emergency  and  non-emergency  conditions. 
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Part  2  -  Work  Activity  Categories 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Operate  and  maintain  firearms  and  personal  weapons  (e.g., 
grenade  launchers,  grenades). 

1 

Select  and  prepare  firing  positions  for  firearms  and  personal 
weapons  (e.g.,  grenade  launchers,  grenades). 

2 

Mount  and  dismount  sights,  lights,  and  other  accessories  on  firearms 
and  personal  weapons  (e.g.,  grenade  launchers,  grenades). 

3 

Mount  and  dismount  machine  guns  or  other  personal  weapons  on 
tripods. 

4 

Construct  field-expedient  firing  aids  for  firearms  and  personal 
weapons. 

5 

Load,  unload,  and  correct  malfunctions  in  firearms  and  personal 
weapons. 

6 

Aim,  track,  and  fire  firearms  or  personal  weapons  at  targets, 
stationary  or  moving. 

7 

Coordinate  the  firing  of  firearms  or  personal  weapons  with  the 
actions  or  movement  of  others. 

8 

Perform  routine  maintenance  checks  and  services  on  firearms  and 
personal  weapons. 

9 

Troubleshoot  firearm  and  personal  weapon  malfunctions  and 
misfires. 

B 

Operate  and  maintain  armored  vehicle-mounted  weapons 
(e.g.,  machine  guns,  rocket  launchers). 

10 

Mount  and  dismount  weapon  (e.g.,  machine  gun,  rocket  launcher) 
on  armored  vehicle. 

11 

Load,  unload,  and  correct  malfunctions  in  armored  vehicle- 
mounted  weapon  (e.g.,  machine  gun,  rocket  launcher). 

12 

Aim,  track,  and  fire  armored  vehicle-mounted  weapon  at  targets, 
stationary  or  moving. 

13 

Coordinate  the  firing  of  armored  vehicle-mounted  weapon  with 
the  actions  or  movement  of  others. 

14 

Perform  routine  maintenance  checks  and  services  on  vehicle- 
mounted  weapon. 

15 

Troubleshoot  vehicle-mounted  weapon  malfunctions  and  misfires. 

C 

Engage  in  close  combat  or  self-defense. 

17 

Engage  enemy  in  hand. 

18 

Engage  or  defend  self  against  enemy  with  bayonet  or  other  hand. 

D 

Navigate  and  move  in  the  field. 

19 

Determine  the  direction  or  location  of  self  and  others  (friendly  or 
foe),  with  or  without  the  aid  of  maps  or  other  navigational 
equipment  (e.g.,  compass,  GPS). 

20 

Navigate  from  point  to  point  using  maps  and  navigational 
equipment  (e.g.,  compass,  GPS),  day  or  night. 

21 

Camouflage  or  hide  self  and  equipment  to  avoid  detection. 

22 

Practice  noise,  light,  and  litter  discipline  to  conceal  movement  or 
location. 

23 

Move  self,  weapons,  gear,  or  equipment  through  the  field  and 
around  obstacles  (natural  or  man-made),  under  fire,  day  or  night. 
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No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

24 

Locate,  recover,  and  neutralize  antipersonnel  mines  or  other 
unexploded  ordinance. 

25 

React  to  counterattack,  ambush,  or  other  defensive  combat 
situations  during  movement. 

E 

Drive  and  maintain  armored  wheeled  and  tracked  vehicles. 

26 

Prepare  armored  wheeled  and  tracked  vehicles  for  mission. 

27 

Drive  armored  wheeled  vehicles  alone  or  in  a  convoy  on  a  tactical 
or  non-tactical  mission. 

28 

Drive  armored  tracked  vehicle  alone  or  in  a  convoy  on  a  tactical  or 
non-tactical  mission. 

29 

Perform  routine  maintenance  checks  and  services  on  armored 
wheeled  or  tracked  vehicle. 

F 

Recover  and  tow  armored  wheeled  and  tracked  vehicles. 

30 

Recover  disabled  armored  vehicle  (wheeled  or  tracked)  in  the  field 
using  available  resources  and  equipment. 

31 

Prepare  and  tow  disabled  armored  vehicle  (wheeled  or  tracked) 
using  standard  equipment  and  vehicles. 

G 

Set-up,  operate,  and  maintain  secure  communications  equipment. 

32 

Set-up  and  install  secure  ground  and  mobile  communications 
equipment  (e.g.,  ground  and  airborne  radios,  tactical  telephones). 

33 

Operate  ground  and  mobile  communications  equipment  (e.g., 
ground  and  airborne  radios,  tactical  telephones). 

34 

Perform  routine  maintenance  checks  and  services  on  mobile 
communications  equipment  (e.g.,  ground  and  airborne  radios, 
tactical  telephones). 

35 

Troubleshoot  mobile  communications  equipment  (e.g.,  ground  and 
airborne  radios,  tactical  telephones). 

36 

Identify  and  respond  to  an  electronic  attack  and  other  security 
threats  to  a  secure  communications  network. 

37 

Encode  and  decode  messages  delivered  over  secure  ground  and 
mobile  communications  equipment  (e.g.,  ground  and  airborne 
radios,  tactical  telephones). 

H 

Protect  against  chemical,  biological,  radiological  and  nuclear 
(CBRN)  attacks  or  hazards. 

38 

Prepare  for  or  react  to  a  CBRN  attack  or  hazard  by  protecting  self 
and  others  using  appropriate  gear  or  equipment. 

39 

Detect,  monitor,  or  measure  CBRN  agents  or  levels. 

40 

Decontaminate  self  or  equipment  using  decontamination  kits. 

I 

Perform  first  aid  and  treat  casualties. 

41 

Evaluate  a  casualty's  vital  signs,  status,  and  severity  of  their 
injuries  (e.g.,  blood  loss,  shock). 

42 

Perform  CPR,  prevent  shock,  clear  the  throat  of  a  casualty,  or 
perform  other  basic  first  aid. 

43 

Perform  first  aid  for  an  open  wound  or  other  injuries  (e.g.,  bums, 
bone  fractures)  received  during  combat. 

44 

Perform  first  aid  for  an  open  wound  or  other  injuries  (e.g.,  bums, 
bone  fractures)  received  in  a  non-combat  situation. 

45 

Perform  first  aid  for  a  CBRN  casualty. 
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No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

G-5) 

J 

Evacuate  casualties  or  remains. 

46 

Administer  an  IV  and  stabilize  casualty  for  transport. 

47 

Transport  a  casualty,  on  foot  or  using  a  vehicle  or  other  transport 
equipment. 

48 

Request  a  medical  evacuation. 

49 

Recover,  evacuate,  or  inter  remains. 

K 

Set-up,  operate,  and  maintain  secure  communications  equipment. 

50 

Set-up  and  install  secure  ground  and  mobile  communications 
equipment  (e.g.,  ground  and  airborne  radios,  tactical  telephones). 

51 

Operate  ground  and  mobile  communications  equipment  (e.g., 
ground  and  airborne  radios,  tactical  telephones). 

52 

Perform  routine  maintenance  checks  and  services  on  mobile 
communications  equipment  (e.g.,  ground  and  airborne  radios, 
tactical  telephones). 

53 

Troubleshoot  mobile  communications  equipment  (e.g.,  ground  and 
airborne  radios,  tactical  telephones). 

54 

Identify  and  respond  to  an  electronic  attack  and  other  security 
threats  to  a  secure  communications  network. 

55 

Encode  and  decode  messages  delivered  over  secure  ground  and 
mobile  communications  equipment  (e.g.,  ground  and  airborne 
radios,  tactical  telephones). 

L 

Guard,  search,  and  monitor  detainees  or  prisoners. 

56 

Guard  or  protect  detainees  or  prisoners. 

57 

Search  detainees  or  prisoners  for  weapons  or  prohibited  items. 

58 

Restrain  or  subdue  detainees  or  prisoners  using  the  appropriate 
level  of  force. 

59 

Monitor  detainee  or  prisoner  activity. 

60 

Escort  detainees  or  prisoners  to  a  new  location,  inside  or  outside  a 
detainment  or  corrections  facility. 

M 

Guard,  search,  or  patrol  areas  or  locations. 

61 

Guard  or  patrol  an  assigned  area(s)  or  location)  s). 

62 

Search  an  area  or  location  for  a  person(s)  or  object)  s). 

63 

Obtain  information  from  the  general  public  or  potential  witnesses 
on  suspicious  activity  in  area  or  location. 

64 

Apprehend,  search,  and  detain  a  person) s)  for  suspicious  activity. 

65 

Restrain  or  subdue  a  person) s)  using  the  appropriate  level  of  force. 

66 

Defuse  a  situation) s)  using  interpersonal  skills. 

N 

Guard  and  control  vehicle  and  pedestrian  traffic. 

67 

Guard  or  patrol  key  access  points  or  entrances  to  restricted  areas 
(e.g.,  dismount  points,  roadblocks,  or  checkpoints). 

68 

Search  or  inspect  vehicles  or  persons  for  explosive  devices  or 
prohibited  items. 

69 

Direct  or  control  vehicle  or  pedestrian  traffic. 

70 

Monitor  vehicle  or  pedestrian  traffic  for  suspicious  activity. 

71 

Direct  or  control  the  behavior  of  groups  or  crowds  of  non- 
combatants,  under  emergency  and  non-emergency  conditions. 
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Part  3  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

1 .  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

2.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

3.  Are  there  work  activities  important  to  the  job  of  first-term  Soldiers  in  your  MOS  that  were 
not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

4.  Please  enter  any  additional  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Army  Work  Activities  Questionnaire 

19K  Ml  Abrams  Tank  Armor  Crewman 
Instructions 

The  purpose  of  this  questionnaire  is  to  find  out  about  the  work  activities  performed  by  first-term 
Soldiers  in  your  MOS.  As  you  complete  this  questionnaire,  consider  the  typical  first-term  Soldier 
in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a 
typical  unit  assignment  and  position  and  at  least  one  series  of  Green- Amber-Red  training  cycles 
and  unit  operational  requirements. 

This  questionnaire  consists  of  a  series  of  work  activities  describing  what  first-term  Soldiers  in  your 
MOS  do,  to  whom  or  what  it  is  done,  and  potentially  for  what  purpose  it  is  done.  For  example: 


Operate  radio  communications  equipment. 


For  each  work  activity,  you  will  be  asked  to  make  three  ratings. 

The  first  involves  rating  each  work  activity  in  terms  of  its  importance  to  the  job  of first-term 
Soldiers  in  your  MOS.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Important 

Somewhat 

Important 

Important 

Very  Important 

Extremely 

Important 

1 

2 

3 

4 

5 

The  second  involves  rating  each  work  activity  in  tenns  of  how  difficult  it  is  for  first  term  Soldiers  in 
your  MOS  to  learn  to  perform  the  activity  to  standard  through  classroom  training.  You  will  use  the 
following  scale  to  make  this  rating: 


Very  Easy  to 
Learn 

Somewhat  Easy 
to  Learn 

Neither  Easy 
nor  Difficult  to 
Learn 

Somewhat 
Difficult  to 
Learn 

Very  Difficult  to 
Learn 

1 

2 

3 

4 

5 

The  third  involves  rating  each  work  activity  in  terms  of  how  serious  the  negative  consequences 
would  be  for  the  Soldier  ’ s  unit  or  the  unit ’s  mission  if  a  first-term  Soldier  in  your  MOS  failed  to 
successfully  perform  the  activity.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Serious 

Somewhat 

Serious 

Serious 

Very  Serious 

Extremely 

Serious 

1 

2 

3 

4 

5 

You  will  complete  this  set  of  ratings  twice.  You  will  first  rate  all  of  the  individual  work 
activities.  You  will  then  rate  the  higher-level  categories  these  work  activities  fall  under. 
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Part  1  -  Individual  Work  Activities 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Operate  and  maintain  radio  communications 
equipment. 

1 

Install  and  configure  radio  communications  equipment  for 
use  in  tank. 

2 

Operate  radio  communications  equipment. 

3 

Perform  routine  maintenance  checks  and  services  on  radio 
communications  equipment. 

4 

Troubleshoot  radio  communications  equipment. 

B 

Operate  and  maintain  tank  computer  and  digital 
information  systems. 

5 

Install  software  and  upload  data  on  tank  computer  and 
digital  information  systems. 

6 

Operate  tank  computer  and  digital  information  systems. 

7 

Perform  routine  maintenance  checks  and  services  on  tank 
computer  and  digital  information  systems. 

C 

Drive  tank. 

8 

Prepare  the  tank  driver’s  station  for  operation. 

9 

Perform  tank  driver’s  operation  checks  and  services. 

10 

Drive  tank. 

11 

Troubleshoot  tank  malfunctions  using  driver’s  control 
panel  and  digital  warning  messages. 

D 

Operate  and  maintain  tank  main  gun. 

12 

Prepare  the  tank  loader's  station  for  operation. 

13 

Load  and  unload  tank  main  gun. 

14 

Perform  routine  maintenance  and  services  on  tank  main 
gun. 

15 

Troubleshoot  tank  main  gun  malfunctions  and  misfires. 

E 

Operate  and  maintain  tank-mounted  machine  guns. 

16 

Mount  and  dismount  machine  guns  from  tank. 

17 

Load  and  unload  tank-mounted  machine  guns. 

18 

Aim,  track,  and  fire  tank-mounted  machine  guns  at  targets. 

19 

Perform  routine  maintanence  checks  and  services  on  tank- 
mounted  machine  guns. 

20 

Troubleshoot  tank-mounted  machine  gun  malfunctions  and 
misfires. 

F 

Recover  and  tow  tank. 

21 

Recover  disabled  tank  in  the  field  using  available  resources 
and  equipment. 

22 

Prepare  and  tow  disabled  tank  using  standard  equipment 
and  vehicles. 
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No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

F 

Operate  and  maintain  tank  mounted  mine  clearing 
equipment. 

23 

Perform  routine  maintenance  checks  and  services  on  tank- 
mounted  mine  clearing  equipment. 

24 

Operate  tank-mounted  mine  clearing  equipment. 

G 

Maintain  and  service  non-weapons  tank  systems  and 
equipment. 

25 

Perform  routine  maintenance  checks  and  services  on  tank 
automotive  system. 

26 

Perform  routine  maintenance  checks  on  basic  issue  tank 
equipment,  such  as  portable  fire  extinguishers. 

27 

Decontaminate  tank  equipment  using  portable 
decontamination  equipment. 

H 

Inspect  and  stow  tank  gun  ammunition. 

28 

Inspect  tank  gun  ammunition  prior  to  stowing. 

29 

Prepare  and  stow  ammunition  in  tank. 

I 

Handle  and  evacuate  casualties. 

30 

Evacuate  a  wounded  crewman  from  a  tank. 

31 

Request  a  medical  evacuation. 

J 

Navigate  in  the  field. 

32 

Assemble  and  read  maps. 

33 

Guide  tank  movements  for  driver  using  hand  signals  and  a 
flashlight. 
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Part  2  -  Work  Activity  Categories 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Operate  and  maintain  radio  communications 
equipment. 

1 

Install  and  configure  radio  communications  equipment  for 
use  in  tank. 

2 

Operate  radio  communications  equipment. 

3 

Perform  routine  maintenance  checks  and  services  on  radio 
communications  equipment. 

4 

Troubleshoot  radio  communications  equipment. 

B 

Operate  and  maintain  tank  computer  and  digital 
information  systems. 

5 

Install  software  and  upload  data  on  tank  computer  and 
digital  information  systems. 

6 

Operate  tank  computer  and  digital  information  systems. 

7 

Perform  routine  maintenance  checks  and  services  on  tank 
computer  and  digital  information  systems. 

C 

Drive  tank. 

8 

Prepare  the  tank  driver’s  station  for  operation. 

9 

Perform  tank  driver’s  operation  checks  and  services. 

10 

Drive  tank. 

11 

Troubleshoot  tank  malfunctions  using  driver’s  control 
panel  and  digital  warning  messages. 

D 

Operate  and  maintain  tank  main  gun. 

12 

Prepare  the  tank  loader's  station  for  operation. 

13 

Load  and  unload  tank  main  gun. 

14 

Perform  routine  maintenance  and  services  on  tank  main 
gun. 

15 

Troubleshoot  tank  main  gun  malfunctions  and  misfires. 

E 

Operate  and  maintain  tank-mounted  machine  guns. 

16 

Mount  and  dismount  machine  guns  from  tank. 

17 

Load  and  unload  tank-mounted  machine  guns. 

18 

Aim,  track,  and  fire  tank-mounted  machine  guns  at  targets. 

19 

Perform  routine  maintanence  checks  and  services  on  tank- 
mounted  machine  guns. 

20 

Troubleshoot  tank-mounted  machine  gun  malfunctions  and 
misfires. 

F 

Recover  and  tow  tank. 

21 

Recover  tank  in  the  field  using  available  resources  and 
equipment. 

22 

Prepare  and  tow  tank  using  standard  equipment  and 
vehicles. 
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No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

F 

Operate  and  maintain  tank  mounted  mine  clearing 
equipment. 

23 

Perform  routine  maintenance  checks  and  services  on  tank- 
mounted  mine  clearing  equipment. 

24 

Operate  tank-mounted  mine  clearing  equipment. 

G 

Maintain  and  service  non-weapons  tank  systems  and 
equipment. 

25 

Perform  routine  maintenance  checks  and  services  on  tank 
automotive  system. 

26 

Perform  routine  maintenance  checks  on  basic  issue  tank 
equipment,  such  as  portable  fire  extinguishers. 

27 

Decontaminate  tank  equipment  using  portable 
decontamination  equipment. 

H 

Inspect  and  stow  tank  gun  ammunition. 

28 

Inspect  tank  gun  ammunition  prior  to  stowing. 

29 

Prepare  and  stow  ammunition  in  tank. 

I 

Handle  and  evacuate  casualties. 

30 

Evacuate  a  wounded  crewman  from  a  tank. 

31 

Request  a  medical  evacuation. 

J 

Navigate  in  the  field. 

32 

Assemble  and  read  maps. 

33 

Guide  tank  movements  for  driver  using  hand  signals  and  a 
flashlight. 
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Part  3  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

5.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

6.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

7.  Are  there  work  activities  important  to  the  job  of  first-term  Soldiers  in  your  MOS  that  were 
not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

8.  Please  enter  any  additional  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Army  Work  Activities  Questionnaire 

25U  Signal  Support  Systems  Specialist 
Instructions 

The  purpose  of  this  questionnaire  is  to  find  out  about  the  work  activities  performed  by  first-term 
Soldiers  in  your  MOS.  As  you  complete  this  questionnaire,  consider  the  typical  first-term  Soldier 
in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a 
typical  unit  assignment  and  position  and  at  least  one  series  of  Green- Amber-Red  training  cycles 
and  unit  operational  requirements. 

This  questionnaire  consists  of  a  series  of  work  activities  describing  what  first-term  Soldiers  in  your 
MOS  do,  to  whom  or  what  it  is  done,  and  potentially  for  what  purpose  it  is  done.  For  example: 


Operate  radio  communications  equipment. 


For  each  work  activity,  you  will  be  asked  to  make  three  ratings. 

The  first  involves  rating  each  work  activity  in  terms  of  its  importance  to  the  job  of first-term 
Soldiers  in  your  MOS.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Important 

Somewhat 

Important 

Important 

Very  Important 

Extremely 

Important 

1 

2 

3 

4 

5 

The  second  involves  rating  each  work  activity  in  terms  of  how  difficult  it  is  for  first  term  Soldiers  in 
your  MOS  to  learn  to  perform  the  activity  to  standard  through  classroom  training.  You  will  use  the 
following  scale  to  make  this  rating: 


Ver\>  Easy  to 
Learn 

Somewhat  Easy 
to  Learn 

Neither  Easy 
nor  Difficult  to 
Learn 

Somewhat 
Difficult  to 
Learn 

Very  Difficult  to 
Learn 

1 

2 

3 

4 

5 

The  third  involves  rating  each  work  activity  in  terms  of  how  serious  the  negative  consequences 
would  be  for  the  Soldier ’s  unit  or  the  unit ’s  mission  if  a  first-term  Soldier  in  your  MOS  failed  to 
successfully  perform  the  activity.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Serious 

Somewhat 

Serious 

Serious 

Very  Serious 

Extremely 

Serious 

1 

2 

3 

4 

5 

You  will  complete  this  set  of  ratings  twice.  You  will  first  rate  all  of  the  individual  work 
activities.  You  will  then  rate  the  higher-level  categories  these  work  activities  fall  under. 
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Part  1  -  Individual  Work  Activities 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Install,  maintain,  and  troubleshoot  commercial  computers 
and  peripherals. 

1 

Set-up  and  configure  commercial  desktop  or  laptop  computers 
and  peripherals  such  as  printers. 

2 

Install  network  hardware  and  software  in  commercial  desktop 
or  laptop  computers. 

3 

Perform  scheduled  maintenance  and  services  on  commercial 
desktop  or  laptop  computers. 

4 

Troubleshoot  and  repair  commercial  desktop  or  laptop 
computers  to  operation. 

B 

Install,  maintain,  and  troubleshoot  secure  data 
transmission  equipment. 

5 

Set-up  and  install  secure  data  transmission  equipment. 

6 

Perform  scheduled  maintenance  and  services  on  secure  data 
transmission  equipment. 

7 

Troubleshoot  and  repair  secure  data  transmission  equipment  to 
operation. 

C 

Install,  maintain,  and  troubleshoot  tactical  computer 
networks  and  systems. 

8 

Install  tactical  computer  networks  and  systems  such  as  the 

Force  XXI  Battle  Command  Brigade  and  Below  (FBCB2). 

9 

Perform  scheduled  maintenance  and  services  on  tactical 
computer  networks  and  systems  such  as  the  Force  XXI  Battle 
Command  Brigade  and  Below  (FBCB2). 

10 

Troubleshoot  and  repair  tactical  computer  networks  and 
systems  such  as  the  Force  XXI  Battle  Command  Brigade  and 
Below  (FBCB2)  to  operation. 

D 

Install,  maintain,  and  troubleshoot  secure  communications 
and  tactical  satellite  systems. 

11 

Set-up  and  install  secure  telephones. 

12 

Troubleshoot  and  repair  secure  telephones  to  operation. 

13 

Install  secure  radio  equipment  and  mobile  communications 
systems. 

14 

Perform  scheduled  maintenance  and  services  on  secure  radio 
equipment  and  mobile  communications  systems. 

15 

Troubleshoot  and  repair  secure  radio  equipment  and  mobile 
communications  systems  to  operation. 

16 

Install  tactical  satellite  equipment. 

17 

Perform  scheduled  maintenance  and  services  on  tactical 
satellite  equipment. 

18 

Troubleshoot  and  repair  tactical  satellite  equipment  to 
operation. 

19 

Install,  splice,  and  knot  wire  and  cable. 
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No. 

Work  Activity 

Importance 

d-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

E 

Install,  maintain,  and  troubleshoot  communication 
security  (COMSEC)  equipment. 

20 

Set-up  and  install  secure  satellite  equipment  such  as  AN/VRC- 
49  and  AN/VRC-92  at  a  designated  field  site  to  transmit 
messages  between  radio  stations. 

21 

Monitor  secure  satellite  equipment  at  designated  field  site  for 
correct  operation. 

22 

Operate  secure  satellite  equipment  at  designated  field  site. 

23 

Monitor  and  identify  potential  security  threats  to 
communication  networks  and  systems. 

24 

Set-up  passwords  and  install  communication  security 
(COMSEC)  equipment. 

25 

Perform  scheduled  maintenance  and  services  on 
communication  security  (COMSEC)  equipment. 

26 

Troubleshoot  and  repair  communication  security  (COMSEC) 
equipment  to  operation. 

F 

Provide  instruction  and  technical  support  to  users  of 
computer  and  communications  equipment. 

27 

Obtain  information  on  computer  and  communications 
equipment  from  users. 

28 

Instruct  users  on  the  operation  of  computer  and 
communications  equipment. 

29 

Provide  technical  support  on  computer  and  communications 
equipment  to  users. 

30 

Understand  technical  operating,  service  or  repair  manuals  for 
computer  and  communications  equipment. 

G 

Complete  records  and  forms  on  computer  and 
communications  equipment. 

31 

Complete  maintenance  records  and  forms  on  computer  and 
communications  equipment. 
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Part  2  -  Work  Activity  Categories 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Install,  maintain,  and  troubleshoot  commercial  computers 
and  peripherals. 

1 

Set-up  and  configure  commercial  desktop  or  laptop  computers 
and  peripherals  such  as  printers. 

2 

Install  network  hardware  and  software  in  commercial  desktop 
or  laptop  computers. 

3 

Perform  scheduled  maintenance  and  services  on  commercial 
desktop  or  laptop  computers. 

4 

Troubleshoot  and  repair  commercial  desktop  or  laptop 
computers  to  operation. 

B 

Install,  maintain,  and  troubleshoot  secure  data 
transmission  equipment. 

5 

Set-up  and  install  secure  data  transmission  equipment. 

6 

Perform  scheduled  maintenance  and  services  on  secure  data 
transmission  equipment. 

7 

Troubleshoot  and  repair  secure  data  transmission  equipment  to 
operation. 

C 

Install,  maintain,  and  troubleshoot  tactical  computer 
networks  and  systems. 

8 

Install  tactical  computer  networks  and  systems  such  as  the 

Force  XXI  Battle  Command  Brigade  and  Below  (FBCB2). 

9 

Perform  scheduled  maintenance  and  services  on  tactical 
computer  networks  and  systems  such  as  the  Force  XXI  Battle 
Command  Brigade  and  Below  (FBCB2). 

10 

Troubleshoot  and  repair  tactical  computer  networks  and 
systems  such  as  the  Force  XXI  Battle  Command  Brigade  and 
Below  (FBCB2)  to  operation. 

D 

Install,  maintain,  and  troubleshoot  secure  communications 
and  tactical  satellite  systems. 

11 

Set-up  and  install  secure  telephones. 

12 

Troubleshoot  and  repair  secure  telephones  to  operation. 

13 

Install  secure  radio  equipment  and  mobile  communications 
systems. 

14 

Perform  scheduled  maintenance  and  services  on  secure  radio 
equipment  and  mobile  communications  systems. 

15 

Troubleshoot  and  repair  secure  radio  equipment  and  mobile 
communications  systems  to  operation. 

16 

Install  tactical  satellite  equipment. 

17 

Perform  scheduled  maintenance  and  services  on  tactical 
satellite  equipment. 

18 

Troubleshoot  and  repair  tactical  satellite  equipment  to 
operation. 

19 

Install,  splice,  and  knot  wire  and  cable. 
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No. 

Work  Activity 

Importance 

d-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

E 

Install,  maintain,  and  troubleshoot  communication 
security  (COMSEC)  equipment. 

20 

Set-up  and  install  secure  satellite  equipment  such  as  AN/VRC- 
49  and  AN/VRC-92  at  a  designated  field  site  to  transmit 
messages  between  radio  stations. 

21 

Monitor  secure  satellite  equipment  at  designated  field  site  for 
correct  operation. 

22 

Operate  secure  satellite  equipment  at  designated  field  site. 

23 

Monitor  and  identify  potential  security  threats  to 
communication  networks  and  systems. 

24 

Set-up  passwords  and  install  communication  security 
(COMSEC)  equipment. 

25 

Perform  scheduled  maintenance  and  services  on 
communication  security  (COMSEC)  equipment. 

26 

Troubleshoot  and  repair  communication  security  (COMSEC) 
equipment  to  operation. 

F 

Provide  instruction  and  technical  support  to  users  of 
computer  and  communications  equipment. 

27 

Obtain  information  on  computer  and  communications 
equipment  from  users. 

28 

Instruct  users  on  the  operation  of  computer  and 
communications  equipment. 

29 

Provide  technical  support  on  computer  and  communications 
equipment  to  users. 

30 

Understand  technical  operating,  service  or  repair  manuals  for 
computer  and  communications  equipment. 

G 

Complete  records  and  forms  on  computer  and 
communications  equipment. 

31 

Complete  maintenance  records  and  forms  on  computer  and 
communications  equipment. 
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Part  3  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

9.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

10.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

11.  Are  there  work  activities  important  to  the  job  of  first-term  Soldiers  in  your  MOS  that  were 
not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

12.  Please  enter  any  additional  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Army  Work  Activities  Questionnaire 

3 IB  Military  Police 
Instructions 


The  purpose  of  this  questionnaire  is  to  find  out  about  the  work  activities  performed  by  first-term 
Soldiers  in  your  MOS.  As  you  complete  this  questionnaire,  consider  the  typical  first-term  Soldier 
in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a 
typical  unit  assignment  and  position  and  at  least  one  series  of  Green- Amber-Red  training  cycles 
and  unit  operational  requirements. 

This  questionnaire  consists  of  a  series  of  work  activities  describing  what  first-term  Soldiers  in  your 
MOS  do,  to  whom  or  what  it  is  done,  and  potentially  for  what  purpose  it  is  done.  For  example: 


Operate  radio  communications  equipment. 


For  each  work  activity,  you  will  be  asked  to  make  three  ratings. 

The  first  involves  rating  each  work  activity  in  terms  of  its  importance  to  the  job  of first-term 
Soldiers  in  your  MOS.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Important 

Somewhat 

Important 

Important 

Ven’  Important 

Extremely 

Important 

1 

2 

3 

4 

5 

The  second  involves  rating  each  work  activity  in  terms  of  how  difficult  it  is  for  first  term  Soldiers  in 
your  MOS  to  learn  to  perform  the  activity  to  standard  through  classroom  training.  You  will  use  the 
following  scale  to  make  this  rating: 


Very  Easy  to 
Learn 

Somewhat  Easy 
to  Learn 

Neither  Easy 
nor  Difficult  to 
Learn 

Somewhat 
Difficult  to 
Learn 

Very  Difficult  to 
Learn 

1 

2 

3 

4 

5 

The  third  involves  rating  each  work  activity  in  terms  of  how  serious  the  negative  consequences 
would  be  for  the  Soldier ’s  unit  or  the  unit ’s  mission  if  a  first-term  Soldier  in  your  MOS  failed  to 
successfully  perform  the  activity.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Serious 

Somewhat 

Serious 

Serious 

Very  Serious 

Extremely 

Serious 

1 

2 

3 

4 

5 

You  will  complete  this  set  of  ratings  twice.  You  will  first  rate  all  of  the  individual  work 
activities.  You  will  then  rate  the  higher-level  categories  these  work  activities  fall  under. 
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Part  1  -  Individual  Work  Activities 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Conduct  law  enforcement  activities. 

1 

Guard  or  patrol  a  designated  area(s)  or  location(s). 

2 

Search  an  area  or  location  for  an  individual(s)  or  object(s). 

3 

Obtain  information  from  potential  witnesses  or  the  general 
public  on  suspicious  activity  in  area  or  location. 

4 

Respond  to  and  investigate  complaints  or  disturbances. 

5 

Apprehend,  search,  and  arrest  a  criminal  suspect(s). 

6 

Restrain  or  subdue  a  criminal  suspect(s)  using  the 
appropriate  level  of  force. 

7 

Serve  arrest,  search  or  seizure  warrants. 

8 

Serve  or  issue  summonses  or  subpoenas. 

9 

Defuse  a  situation(s)  using  interpersonal  skills. 

B 

Control  and  enforce  traffic  laws. 

10 

Guard  or  patrol  entrances  at  key  locations  (e.g.,  dismount 
points,  roadblocks,  or  checkpoints). 

11 

Search  or  inspect  vehicles  for  explosive  devices  or 
prohibited  items. 

12 

Direct  or  control  vehicle  or  pedestrian  traffic. 

13 

Monitor  vehicle  or  pedestrian  traffic  for  suspicious  activity. 

14 

Issue  citation  or  warning  for  violation  of  traffic  laws. 

15 

Respond  to  and  investigate  traffic  violations  or  accidents. 

16 

Direct  or  control  groups  or  crowds  of  non-combatants,  under 
emergency  and  non-emergency  conditions. 

C 

Guard  and  monitor  detainees  or  prisoners. 

17 

Guard  or  protect  detainees  or  prisoners. 

18 

Search  detainees  or  prisoners  for  weapons  or  prohibited 
items. 

19 

Restrain  or  subdue  detainees  or  prisoners  using  the 
appropriate  level  of  force. 

20 

Monitor  detainee  or  prisoner  activity. 

21 

Escort  detainees  or  prisoners  to  a  new  location,  inside  or 
outside  a  detainment  or  corrections  facility. 

22 

Respond  to  emergency  situations  (e.g.,  fights,  riots,  escapes) 
in  detainment  or  corrections  facility. 

23 

Guard  or  patrol  area  or  premises  in  a  detainment  or 
corrections  facility. 

24 

Search  or  inspect  area  or  premises  in  a  detainment  or 
corrections  facility. 

25 

Search  or  inspect  incoming  and  outgoing  materials  for 
detainees  or  prisoners. 

D 

Assist  in  crime  scene  investigations. 

26 

Establish  a  perimeter  for  crime  scene  and  control  access. 

27 

Guard  or  secure  evidence  at  or  from  a  crime  scene. 

28 

Gather  or  collect  evidence  at  crime  scene. 
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No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

29 

Interview  crime  victims,  witnesses,  or  suspects  to  obtain 
relevant  information. 

E 

Guard  or  patrol  people,  places,  or  objects  as  member  of 
a  security  team. 

30 

Guard  or  patrol  a  facility  as  member  of  a  team. 

31 

Guard  or  protect  individual(s)  as  member  of  a  team. 

32 

Guard  or  patrol  cargo  as  a  member  of  a  team. 

33 

Guard  or  secure  classified  information  or  materials  during 
transport  as  member  of  a  team. 

F 

Set-up  and  operate  radio  communications  equipment. 

34 

Operate  radio  communications  equipment. 

35 

Set-up  a  secure  radio  communications  site. 

36 

Identify  and  respond  to  an  electronic  attack  and  other 
security  threats  over  a  radio  communications  network. 

G 

Prepare  forms  and  reports. 

37 

Prepare  initial  criminal  incident  reports. 

38 

Prepare  and  maintain  records  and  logs  on  detainees  or 
prisoners. 

39 

Prepare  CBRN,  spot,  and  situational  reports. 

H 

Drive  wheeled  vehicles. 

40 

Drive  armored  wheeled  vehicles  (e.g.,  ASV)  alone  or  in  a 
convoy  on  a  non-tactical  mission. 

41 

Drive  armored  wheeled  vehicle  (e.g.,  ASV)  alone  or  in  a 
convoy  on  a  tactical  mission. 

42 

Follow  traffic  laws  and  regulations. 

43 

Navigate  from  point  to  point  using  maps  and  navigational 
equipment  (e.g.,  GPS). 
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Part  2  -  Work  Activity  Categories 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Conduct  law  enforcement  activities. 

1 

Guard  or  patrol  a  designated  area(s)  or  location(s). 

2 

Search  an  area  or  location  for  an  individual(s)  or  object(s). 

3 

Obtain  information  from  potential  witnesses  or  the  general  public 
on  suspicious  activity  in  area  or  location. 

4 

Respond  to  and  investigate  complaints  or  disturbances. 

5 

Apprehend,  search,  and  arrest  a  criminal  suspect(s). 

6 

Restrain  or  subdue  a  criminal  suspect(s)  using  the  appropriate 
level  of  force. 

7 

Serve  arrest,  search  or  seizure  warrants. 

8 

Serve  or  issue  summonses  or  subpoenas. 

9 

Defuse  a  situation(s)  using  interpersonal  skills. 

B 

Control  and  enforce  traffic  laws. 

10 

Guard  or  patrol  entrances  at  key  locations  (e.g.,  dismount  points, 
roadblocks,  or  checkpoints). 

11 

Search  or  inspect  vehicles  for  explosive  devices  or  prohibited 
items. 

12 

Direct  or  control  vehicle  or  pedestrian  traffic. 

13 

Monitor  vehicle  or  pedestrian  traffic  for  suspicious  activity. 

14 

Issue  citation  or  warning  for  violation  of  traffic  laws. 

15 

Respond  to  and  investigate  traffic  violations  or  accidents. 

16 

Direct  or  control  groups  or  crowds  of  non-combatants,  under 
emergency  and  non-emergency  conditions. 

C 

Guard  and  monitor  detainees  or  prisoners. 

17 

Guard  or  protect  detainees  or  prisoners. 

18 

Search  detainees  or  prisoners  for  weapons  or  prohibited  items. 

19 

Restrain  or  subdue  detainees  or  prisoners  using  the  appropriate 
level  of  force. 

20 

Monitor  detainee  or  prisoner  activity. 

21 

Escort  detainees  or  prisoners  to  a  new  location,  inside  or  outside  a 
detainment  or  corrections  facility. 

22 

Respond  to  emergency  situations  (e.g.,  fights,  riots,  escapes)  in 
detainment  or  corrections  facility. 

23 

Guard  or  patrol  area  or  premises  in  a  detainment  or  corrections 
facility. 

24 

Search  or  inspect  area  or  premises  in  a  detainment  or  corrections 
facility. 

25 

Search  or  inspect  incoming  and  outgoing  materials  for  detainees 
or  prisoners. 

D 

Assist  in  crime  scene  investigations. 

26 

Establish  a  perimeter  for  crime  scene  and  control  access. 

27 

Guard  or  secure  evidence  at  or  from  a  crime  scene. 

28 

Gather  or  collect  evidence  at  crime  scene. 

29 

Interview  crime  victims,  witnesses,  or  suspects  to  obtain  relevant 
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No. 

Work  Activity 

Importance 

G-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

information. 

E 

Guard  or  patrol  people,  places,  or  objects  as  member  of  a 
security  team. 

30 

Guard  or  patrol  a  facility  as  member  of  a  team. 

31 

Guard  or  protect  individual(s)  as  member  of  a  team. 

32 

Guard  or  patrol  cargo  as  a  member  of  a  team. 

33 

Guard  or  secure  classified  information  or  materials  during 
transport  as  member  of  a  team. 

F 

Set-up  and  operate  radio  communications  equipment. 

34 

Operate  radio  communications  equipment. 

35 

Set-up  a  secure  radio  communications  site. 

36 

Identify  and  respond  to  an  electronic  attack  and  other  security 
threats  over  a  radio  communications  network. 

G 

Prepare  forms  and  reports. 

37 

Prepare  initial  criminal  incident  reports. 

38 

Prepare  and  maintain  records  and  logs  on  detainees  or  prisoners. 

39 

Prepare  CBRN,  spot,  and  situational  reports. 

H 

Drive  wheeled  vehicles. 

40 

Drive  armored  wheeled  vehicles  (e.g.,  ASV)  alone  or  in  a  convoy 
on  a  non-tactical  mission. 

41 

Drive  armored  wheeled  vehicle  (e.g.,  ASV)  alone  or  in  a  convoy 
on  a  tactical  mission. 

42 

Follow  traffic  laws  and  regulations. 

43 

Navigate  from  point  to  point  using  maps  and  navigational 
equipment  (e.g.,  GPS). 
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Part  3  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

13.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

14.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

15.  Are  there  work  activities  important  to  the  job  of  first-term  Soldiers  in  your  MOS  that  were 
not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

16.  Please  enter  any  additional  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Army  Work  Activities  Questionnaire 

63B  Wheeled  Vehicle  Mechanic 

Instructions 

The  purpose  of  this  questionnaire  is  to  find  out  about  the  work  activities  performed  by  first-term 
Soldiers  in  your  MOS.  As  you  complete  this  questionnaire,  consider  the  typical  first-term  Soldier 
in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a 
typical  unit  assignment  and  position  and  at  least  one  series  of  Green- Amber-Red  training  cycles 
and  unit  operational  requirements. 

This  questionnaire  consists  of  a  series  of  work  activities  describing  what  first-term  Soldiers  in  your 
MOS  do,  to  whom  or  what  it  is  done,  and  potentially  for  what  purpose  it  is  done.  For  example: 


Operate  radio  communications  equipment. 


For  each  work  activity,  you  will  be  asked  to  make  three  ratings. 

The  first  involves  rating  each  work  activity  in  terms  of  its  importance  to  the  job  of first-term 
Soldiers  in  your  MOS.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Important 

Somewhat 

Important 

Important 

Very  Important 

Extremely 

Important 

1 

2 

3 

4 

5 

The  second  involves  rating  each  work  activity  in  tenns  of  how  difficult  it  is  for  first  term  Soldiers  in 
your  MOS  to  learn  to  perform  the  activity  to  standard  through  classroom  training.  You  will  use  the 
following  scale  to  make  this  rating: 


Very  Easy  to 
Learn 

Somewhat  Easy 
to  Learn 

Neither  Easy 
nor  Difficult  to 
Learn 

Somewhat 
Difficult  to 
Learn 

Very  Difficult  to 
Learn 

1 

2 

3 

4 

5 

The  third  involves  rating  each  work  activity  in  terms  of  how  serious  the  negative  consequences 
would  be  for  the  Soldier  ’ s  unit  or  the  unit ’s  mission  if  a  first-term  Soldier  in  your  MOS  failed  to 
successfully  perform  the  activity.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Serious 

Somewhat 

Serious 

Serious 

Very  Serious 

Extremely 

Serious 

1 

2 

3 

4 

5 

You  will  complete  this  set  of  ratings  twice.  You  will  first  rate  all  of  the  individual  work 
activities.  You  will  then  rate  the  higher-level  categories  these  work  activities  fall  under. 
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Part  1  -  Individual  Work  Activities 


No. 

Work  Activity 

Importance 

d-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Perform  scheduled  maintenance  checks  and  services  on 

wheeled  vehicle. 

1 

Obtain  service  box  and  read  orders  for  maintenance  job. 

2 

Perform  routine  inspection  and  checks  of  wheeled  vehicle. 

3 

Replace  worn  parts  and  fluids  in  wheeled  vehicle. 

4 

Drive  or  test  wheeled  vehicle  after  completing  scheduled 

maintenance  checks  and  services. 

B 

Troubleshoot  malfunctions  in  wheeled  vehicle. 

5 

Read  and  review  wheeled  vehicle  maintenance  and  service 
records. 

6 

Troubleshoot  malfunctions  in  wheeled  mechanical  and  electrical 
systems. 

7 

Read  and  refer  to  technical  documents  and  manuals. 

C 

Repair  wheeled  vehicle. 

8 

Correct  or  repair  malfunctions  in  wheeled  vehicle  mechanical  and 
electrical  systems. 

9 

Replace  damaged  or  defective  parts  in  wheeled  vehicle 
mechanical  and  electrical  systems. 

10 

Drive  or  test  wheeled  vehicle  after  repair  or  replacement  of  parts. 

D 

Maintain  tools  and  test  equipment. 

11 

Clean  and  maintain  tools  and  test  equipment. 

12 

Secure  and  account  for  tools  and  test  equipment. 

13 

Calibrate  and  service  tools  and  test  equipment. 

E 

Perform  administrative  duties. 

14 

Complete  and  maintain  wheeled  vehicle  maintenance  and  service 

records. 

15 

Prepare  and  submit  fonns  to  order  replacement  parts. 

F 

Prepare  work  area  and  follow  safety  procedures. 

16 

Prepare  work  area  for  wheeled  vehicle  maintenance  and  repair. 

17 

Follow  safety  procedures  when  working  on  wheeled  vehicle. 

18 

Use  protective  clothing  and  special  equipment  when  working 

under  hazardous  conditions. 

C-30 


Part  2  -  Work  Activity  Categories 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

(1-5) 

Critical 

(1-5) 

A 

Perform  scheduled  maintenance  checks  and  services  on 

wheeled  vehicle. 

1 

Obtain  service  box  and  read  orders  for  maintenance  job. 

2 

Perform  routine  inspection  and  checks  of  wheeled  vehicle. 

3 

Replace  worn  parts  and  fluids  in  wheeled  vehicle. 

4 

Drive  or  test  wheeled  vehicle  after  completing  scheduled 

maintenance  checks  and  services. 

B 

Troubleshoot  malfunctions  in  wheeled  vehicle. 

5 

Read  and  review  wheeled  vehicle  maintenance  and  service 
records. 

6 

Troubleshoot  malfunctions  in  wheeled  mechanical  and  electrical 
systems. 

7 

Read  and  refer  to  technical  documents  and  manuals. 

C 

Repair  wheeled  vehicle. 

8 

Correct  or  repair  malfunctions  in  wheeled  vehicle  mechanical  and 
electrical  systems. 

9 

Replace  damaged  or  defective  parts  in  wheeled  vehicle 
mechanical  and  electrical  systems. 

10 

Drive  or  test  wheeled  vehicle  after  repair  or  replacement  of  parts. 

D 

Maintain  tools  and  test  equipment. 

11 

Clean  and  maintain  tools  and  test  equipment. 

12 

Secure  and  account  for  tools  and  test  equipment. 

13 

Calibrate  and  service  tools  and  test  equipment. 

E 

Perform  administrative  duties. 

14 

Complete  and  maintain  wheeled  vehicle  maintenance  and  service 

records. 

15 

Prepare  and  submit  fonns  to  order  replacement  parts. 

F 

Prepare  work  area  and  follow  safety  procedures. 

16 

Prepare  work  area  for  wheeled  vehicle  maintenance  and  repair. 

17 

Follow  safety  procedures  when  working  on  wheeled  vehicle. 

18 

Use  protective  clothing  and  special  equipment  when  working 

under  hazardous  conditions. 
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Part  3  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

17.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

18.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

19.  Are  there  work  activities  important  to  the  job  of  first-term  Soldiers  in  your  MOS  that  were 
not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

20.  Please  enter  any  additional  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Army  Work  Activities  Questionnaire 

88M  Motor  Transport  Operator 
Instructions 

The  purpose  of  this  questionnaire  is  to  find  out  about  the  work  activities  performed  by  first-term 
Soldiers  in  your  MOS.  As  you  complete  this  questionnaire,  consider  the  typical  first-term  Soldier 
in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a 
typical  unit  assignment  and  position  and  at  least  one  series  of  Green- Amber-Red  training  cycles 
and  unit  operational  requirements. 

This  questionnaire  consists  of  a  series  of  work  activities  describing  what  first-term  Soldiers  in  your 
MOS  do,  to  whom  or  what  it  is  done,  and  potentially  for  what  purpose  it  is  done.  For  example: 


Operate  radio  communications  equipment. 


For  each  work  activity,  you  will  be  asked  to  make  three  ratings. 

The  first  involves  rating  each  work  activity  in  terms  of  its  importance  to  the  job  of first-term 
Soldiers  in  your  MOS.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Important 

Somewhat 

Important 

Important 

Very  Important 

Extremely 

Important 

1 

2 

3 

4 

5 

The  second  involves  rating  each  work  activity  in  tenns  of  how  difficult  it  is  for  first  term  Soldiers  in 
your  MOS  to  learn  to  perform  the  activity  to  standard  through  classroom  training.  You  will  use  the 
following  scale  to  make  this  rating: 


Very  Easy  to 
Learn 

Somewhat  Easy 
to  Learn 

Neither  Easy 
nor  Difficult  to 
Learn 

Somewhat 
Difficult  to 
Learn 

Very  Difficult  to 
Learn 

1 

2 

3 

4 

5 

The  third  involves  rating  each  work  activity  in  terms  of  how  serious  the  negative  consequences 
would  be  for  the  Soldier ’s  unit  or  the  unit ’s  mission  if  a  first-term  Soldier  in  your  MOS  failed  to 
successfully  perform  the  activity.  You  will  use  the  following  scale  to  make  this  rating: 


Not  Serious 

Somewhat 

Serious 

Serious 

Very  Serious 

Extremely 

Serious 

1 

2 

3 

4 

5 

You  will  complete  this  set  of  ratings  twice.  You  will  first  rate  all  of  the  individual  work 
activities.  You  will  then  rate  the  higher-level  categories  these  work  activities  fall  under. 
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Part  1  -  Individual  Work  Activities 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

d-5) 

Critical 

d-5) 

A 

Drive  wheeled  vehicle. 

1 

Drive  wheeled  vehicle  alone  or  in  a  convoy  on  a  non-tactical 
mission. 

2 

Drive  wheeled  vehicle  alone  or  in  a  convoy  on  a  tactical  mission. 

3 

Drive  wheeled  vehicle  with  trailer  or  semitrailer. 

4 

Drive  wheeled  vehicle  on  adverse  roads  and  terrain. 

5 

Drive  wheeled  vehicle  under  adverse  climatic  conditions. 

6 

Drive  wheeled  vehicle  at  day  or  night  under  severely  degraded 
visual  conditions. 

7 

Couple  and  uncouple  trailers  or  semitrailers  to  wheeled  vehicle. 

8 

Back  up  wheeled  vehicle  with  trailer  or  semitrailer. 

9 

Follow  traffic  laws  and  regulations. 

B 

Maintain  and  service  wheeled  vehicle. 

10 

Prepare  wheeled  vehicle  for  operations. 

11 

Inspect  wheeled  vehicle  before,  during,  and  after  operations. 

12 

Perform  routine  maintenance  checks  and  services  on  wheeled 
vehicle. 

13 

Understand  wheeled  vehicle  service  or  repair  manuals. 

C 

Transport  personnel  and  cargo. 

14 

Load  or  unload  passengers  for  transport  in  a  wheeled  vehicle. 

15 

Operate  heavy  equipment  to  load  or  unload  cargo  for  transport. 

16 

Manually  load  or  unload  cargo  for  transport  in  a  wheeled  vehicle. 

17 

Secure  and  tiedown  cargo  before  transport  in  wheeled  vehicle. 

18 

Inspect  tiedowns  and  security  of  cargo  before  and  during 
transport  in  wheeled  vehicle. 

D 

Recover  and  tow  wheeled  vehicle. 

19 

Recover  disabled  vehicle  in  the  field  using  available  resources 
and  equipment. 

20 

Prepare  and  tow  disabled  vehicle  using  standard  equipment  and 
vehicles. 

E 

Complete  forms  and  reports. 

21 

Complete  accident  forms. 

22 

Complete  driver  trip  records. 

F 

Navigate  and  steer  wheeled  vehicle  movements. 

23 

Read  maps. 

24 

Operate  vehicle  navigational  and  movement  tracking  systems. 

25 

Steer  the  wheeled  vehicle  movements  of  other  drivers  using  hand 
signals  and  a  flashlight. 
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Part  2  -  Work  Activity  Categories 


No. 

Work  Activity 

Importance 

(1-5) 

Training 

Difficulty 

d-5) 

Critical 

d-5) 

A 

Drive  wheeled  vehicle. 

1 

Drive  wheeled  vehicle  alone  or  in  a  convoy  on  a  non-tactical 
mission. 

2 

Drive  wheeled  vehicle  alone  or  in  a  convoy  on  a  tactical  mission. 

3 

Drive  wheeled  vehicle  with  trailer  or  semitrailer. 

4 

Drive  wheeled  vehicle  on  adverse  roads  and  terrain. 

5 

Drive  wheeled  vehicle  under  adverse  climatic  conditions. 

6 

Drive  wheeled  vehicle  at  day  or  night  under  severely  degraded 
visual  conditions. 

7 

Couple  and  uncouple  trailers  or  semitrailers  to  wheeled  vehicle. 

8 

Back  up  wheeled  vehicle  with  trailer  or  semitrailer. 

9 

Follow  traffic  laws  and  regulations. 

B 

Maintain  and  service  wheeled  vehicle. 

10 

Prepare  wheeled  vehicle  for  operations. 

11 

Inspect  wheeled  vehicle  before,  during,  and  after  operations. 

12 

Perform  routine  maintenance  checks  and  services  on  wheeled 
vehicle. 

13 

Understand  wheeled  vehicle  service  or  repair  manuals. 

C 

Transport  personnel  and  cargo. 

14 

Load  or  unload  passengers  for  transport  in  a  wheeled  vehicle. 

15 

Operate  heavy  equipment  to  load  or  unload  cargo  for  transport. 

16 

Manually  load  or  unload  cargo  for  transport  in  a  wheeled  vehicle. 

17 

Secure  and  tiedown  cargo  before  transport  in  wheeled  vehicle. 

18 

Inspect  tiedowns  and  security  of  cargo  before  and  during 
transport  in  wheeled  vehicle. 

D 

Recover  and  tow  wheeled  vehicle. 

19 

Recover  disabled  vehicle  in  the  field  using  available  resources 
and  equipment. 

20 

Prepare  and  tow  disabled  vehicle  using  standard  equipment  and 
vehicles. 

E 

Complete  forms  and  reports. 

21 

Complete  accident  forms. 

22 

Complete  driver  trip  records. 

F 

Navigate  and  steer  wheeled  vehicle  movements. 

23 

Read  maps. 

24 

Operate  vehicle  navigational  and  movement  tracking  systems. 

25 

Steer  the  wheeled  vehicle  movements  of  other  drivers  using  hand 
signals  and  a  flashlight. 
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Part  3  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

21.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

22.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

23.  Are  there  work  activities  important  to  the  job  of  first-term  Soldiers  in  your  MOS  that  were 
not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

24.  Please  enter  any  additional  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Appendix  D 


Work  Context  Questionnaire 
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Army  Work  Context  Questionnaire 


Instructions 

The  purpose  of  this  questionnaire  is  to  find  out  about  the  kinds  of  conditions  under  which  a  first- 
term  Soldier  in  your  MOS  must  perform.  As  you  complete  this  questionnaire,  consider  the  typical 
first-term  Soldier  in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one 
year  of  duty  in  a  typical  unit  assignment  and  position  and  at  least  one  series  of  Green-Amber- 
Red  training  cycles  and  unit  operational  requirements. 

This  questionnaire  consists  of  a  series  of  statements  describing  different  conditions  under  which 
a  first-term  Soldier  in  your  MOS  must  perform.  For  example: 


Exposed,  or  potentially  exposed,  to  manufactured  (i.e.,  man¬ 
made)  hazards  or  contaminants. 


For  each  statement,  you  will  be  asked  to  make  two  ratings. 

The  first  rating  asks  you  to  rate  how  characteristic  the  statement  is  for  first-term  Soldiers  in  your 
MOS  in  a  non-combat  zone  field  assignment.  When  making  this  rating,  use  the  following  scale: 


Not  at  all 
characteristic 

Slightly 

characteristic 

Somewhat 

characteristic 

Moderately 

characteristic 

Very 

characteristic 

1 

2 

3 

4 

5 

Please  record  your  rating  directly  in  the  space  marked,  “Non-Combat.” 


The  second  rating  asks  you  to  rate  how  characteristic  the  statement  is  for  first-term  Soldiers  in 
your  MOS  in  a  combat  zone  field  assignment  (e.g.,  in  Iraq  or  Afghanistan) .  When  making  this 
rating,  use  the  following  scale: 


Not  at  all 
characteristic 

Slightly 

characteristic 

Somewhat 

characteristic 

Moderately 

characteristic 

Very 

characteristic 

1 

2 

3 

4 

5 

Please  record  your  rating  directly  in  the  space  marked,  “Combat.” 


D-2 


Army  Work  Context  Questionnaire  -  Rating  Sheet 


Part  1  -  Work  Context 

First-term  Soldiers  in  my  MOS  are. . . 

Non- 

Combat  Combat 

#  Item  (1-5)  (1-5) 

1  Exposed,  or  potentially  exposed,  to  manufactured  (i.e.,  man-made) 
hazards  or  contaminants. 

2  Exposed,  or  potentially  exposed,  to  natural  environmental  hazards. 

3  Exposed,  or  potentially  exposed,  to  diseases  or  infection  when 
performing  their  work. 

4  Required  to  operate,  use,  or  handle  hazardous  equipment  or 
materials. 

5  Required  to  work  indoors  or  in  an  environmentally  controlled 
environment. 

6  Required  to  work  outdoors,  exposed  to  all  weather  conditions. 

7  Required  to  work  in  enclosed  or  cramped  spaces. 

8  Exposed  to  sounds  and  noise  levels  that  are  distracting  and 
uncomfortable. 

9  Exposed  to  extreme  temperatures  (hot  or  cold). 

10  Required  to  work  at  night  or  under  inadequate  lighting  conditions. 

1 1  Exposed  to  significant  levels  of  physical  or  mental  discomfort, 
stress,  or  strain. 

12  Required  to  perform  their  work  while  wearing  or  carrying  heavy 
equipment,  supplies,  or  materials. 

13  Required  to  wear  specialized  protective  or  safety  equipment  that 
can  restrict  or  constrain  their  performance. 

14  Required  to  work  with  limited,  inadequate,  or  defective  equipment, 
supplies,  or  materials. 

1 5  Required  to  work  long  hours  or  take  on  additional  duties  due  to 
limited  or  inadequate  personnel. 

1 6  Required  to  perform  their  duties  with  limited  or  inadequate 
information  or  instructions. 

1 7  Required  to  work  through  frequent  or  unscheduled  interruptions 
that  make  it  hard  to  complete  their  work  on  time. 

1 8  Required  to  respond  to  frequent  crises  or  emergencies. 

1 9  Afforded  the  freedom  to  determine  the  timing  and  scheduling  of 
their  work. 
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20 

Afforded  the  freedom  to  determine  which  methods  and  procedures 
are  used  to  complete  their  work. 

21 

Required  to  work  at  a  pace  or  speed  determined  by  equipment  or 
technology. 

22 

Required  to  make  decisions  that  affect,  or  could  affect,  others. 

23 

Required  to  perform  duties  of  long  or  extended  duration. 

24 

Required  to  work  long  or  extended  hours  with  little  to  no  sleep. 

25 

Afforded  limited  rest  or  recovery  time  between  stressful  or 
demanding  tasks. 

26 

Required  to  plan  or  perform  under  significant  time  pressure. 

27 

Required  to  be  very  precise  and  highly  accurate  when  completing 
their  tasks. 

28 

Required  to  perform  under  circumstances  of  conflicting  or 
ambiguous  directions,  orders,  or  priorities. 

29 

Required  to  work  under  minimal  or  limited  supervision. 

30 

Required  to  perform  duties  for  which  they  received  minimal  or 
limited  training. 

31 

Required  to  perform  non-job  specific  duties  or  to  complete  tasks 
outside  of  their  job. 

32 

Required  to  depend  heavily  on  technology  or  equipment  to 
complete  their  tasks. 

33 

Required  to  perform  continuous,  repetitive  physical  or  mental  tasks. 

34 

Required  to  interact  or  deal  with  non-hostile  host  country  nationals. 

35 

Required  to  interact  or  deal  with  hostile  but  non-violent  host 
country  nationals  or  groups. 

36 

Required  to  deal  with  violent  or  physically  aggressive  host  country 
nationals  or  groups. 

37 

Required  to  interact  or  work  with  non- Army  personnel. 

38 

Required  to  perform  work  that  is  highly  visible  to  others  and  for 
which  others  can  observe  what  they  are  doing. 

39 

Required  to  interact  or  work  with  challenging  and  difficult 
coworkers. 

40 

Required  to  work  closely  with  or  depend  heavily  on  others  to 
complete  their  own  duties. 

41 

Required  to  perform  their  work  largely  on  their  own,  with  little 
assistance  from  others. 

42 

Required  to  interact  and  work  with  others  at  a  distance  (e.g., 
through  e-mail  or  other  fonns  of  electronic  communication). 

43 

Required  to  work  as  a  member  of  a  team  consisting  of  individuals 
outside  of  the  Soldier’s  unit  or  the  Army. 

44 

Required  to  coordinate  or  lead  others  for  whom  the  Soldier  has  no 
direct  authority. 
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# 

Item 

Non- 

Combat 

(1-5) 

Combat 

(1-5) 

45 

Required  to  persuade  or  influence  others  for  whom  the  Soldier  has 
no  direct  authority. 

46 

Required  to  be  responsible  for  the  health  and  safety  of  non- Army 
personnel. 
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Army  Work  Context  Questionnaire 


Part  2  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the  following 
questions: 

25.  How  would  you  rate  the  questionnaire  on  the  following: 


a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

26.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any  of 
the  above  questions,  please  explain. 

<SPACE> 

27.  Are  there  conditions  critical  to  your  MOS  under  which  first-term  Soldiers  must  perfonn  that 
were  not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

28.  Please  enter  any  other  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 


D-6 


Appendix  E 


Worker-Oriented  Questionnaires 
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Army  Abilities  Questionnaire 
Instructions 

The  purpose  of  this  questionnaire  is  to  find  out  what  abilities  are  most  important  to  the  work  of  first-term 
Soldiers  in  your  MOS.  Abilities  are  enduring  attributes  of  a  person  that  influence  how  well  they  perform 
their  work.  The  Army  offers  many  different  MOS;  your  answers  will  let  us  know  the  abilities  that  are 
most  descriptive  of  yours.  As  you  complete  this  questionnaire,  consider  the  typical  first-term 
Soldier  in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  has  completed  at  least  one  year  of  duty  in 
a  typical  unit  assignment  and  position  and  at  least  one  series  of  Green-Amber-Red  training 
cycles  and  unit  operational  requirements. 

Rate  each  ability  in  terms  of  its  importance  to  the  work  of  first-term  Soldiers  in  your  MOS.  Y ou  will  use 
the  following  scale  to  make  this  rating: 


1 

2 

3 

4 

5 

Not 

Important 

Somewhat 

Important 

Important 

Very 

Important 

Extremely 

Important 

When  completing  this  questionnaire,  it  is  important  that  you  consider  how  well  each  ability 
describes  the  job  of first-term  Soldiers  in  your  MOS,  not,  the  job  of  NCOs  in  your  MOS. 

CONTINUED  ON  NEXT  PAGE  ^ 


E-2 


PAGE  INTENTIONALLY  LEFT  BLANK 


E-3 


Part  1  -  Abilities 


First-term  Soldiers  in  my  MOS  must  be  able  to... 

Importance 

_ #  Ability _ (1-5) 

1  Read  and  understand  written  documents  and  communicate  with  others  in 
writing. 

2  Listen  to  and  understand  what  others  are  saying  and  speak  effectively  to 
convey  information  to  others. 

3  Come  up  with  unusual  or  creative  ideas  about  a  given  topic  or  generate 
multiple  solutions  to  a  problem. 

4  Tell  when  something  is  wrong  or  is  likely  to  go  wrong. 

5  Use  logic  and  reasoning  to  evaluate  ideas  or  the  strengths  and  weaknesses 
of  potential  solutions  to  problems. 

6  Find  ways  or  use  different  ways  to  structure,  classify,  or  group  multiple 
pieces  of  information  into  a  meaningful  order  or  pattern. 

7  Use  mathematics  to  solve  problems. 

8  Remember  and  recall  infonuation  from  memory  when  needed  (e.g.,  words, 
numbers,  pictures,  and  procedures). 

9  Know  how  to  quickly  find  information  and  make  sense  of  the  information 
when  found. 

10  Quickly  and  accurately  compare  similarities  and  differences  among  objects 
or  stimuli  to  identify  or  detect  a  meaningful  pattern  (e.g.,  a  figure,  object, 
word,  or  sound). 

1 1  Know  their  location  in  relation  to  the  environment  or  know  where  other 
objects  are  in  relation  to  themselves. 

12  Imagine  how  something  will  look  after  it  is  moved  around  or  when  its 
parts  are  moved  or  rearranged. 

1 3  Concentrate  on  a  task  over  a  period  of  time  without  being  distracted. 

14  Perfonu  multiple  activities  simultaneously  or  shift  back  and  forth  between 
two  or  more  sources  of  information  at  once. 

1 5  Learn  new  skills  or  how  to  perform  an  activity  on  their  own,  either  by 
figuring  it  out  themselves  or  by  observing  others. 

1 6  Keep  their  hand  and  arm  steady  while  moving  anu  or  while  holding  arm 
and  hand  in  one  position. 

1 7  Move  fingers,  hands,  or  hands  together  with  arms  to  grasp,  manipulate,  or 
assemble  objects. 

1 8  Use  fingers,  hands,  feet,  or  limbs  to  operate  the  controls  of  a  machine  or 
piece  of  equipment  to  adjust  its  positioning,  speed,  or  direction. 

19  Make  fast,  simple,  repeated  movements  (with  finger,  hand,  foot,  or  limbs) 
in  quick  response  to  a  signal  (e.g.,  sound,  light,  picture)  when  it  appears. 

CONTINUED  ON  NEXT  PAGE  -> 
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# 

Ability 

Importance 

d-5) 

20 

Use  muscles  to  lift,  push,  pull,  carry,  or  throw  objects  or  to  propel  oneself 
(e.g.,  jump  or  sprint). 

21 

Exert  themselves  physically  over  long  periods  of  time  without  getting 
winded  or  out  of  breath  (e.g.,  from  running). 

22 

Bend,  stretch,  twist,  or  reach  out  with  their  body,  arms,  or  legs  (i.e.,  be 
flexible). 

23 

Coordinate  the  movement  of  anns,  legs,  and  torso  together  while 
stationary  or  when  whole  body  is  in  motion. 

24 

Keep  or  regain  their  balance  or  stay  upright  when  in  an  unstable  position. 

25 

See  details  accurately  (e.g.,  of  an  object)  at  close  range  or  at  a  distance. 

26 

Match  or  detect  differences  between  colors,  including  shades  of  color  and 
brightness. 

27 

See  objects  or  movement  of  objects  to  their  side  when  the  eyes  are  looking 
ahead  (i.e.,  peripheral  vision). 

28 

Judge  which  of  several  objects  is  closer  or  farther  away  from  them,  or 
judge  the  distance  between  them  and  an  object. 

29 

Hear  sounds  accurately  and  tell  the  differences  between  sounds  that  vary 
in  pitch  and  loudness. 

30 

Focus  attention  on  a  single  sound  and  tell  from  which  direction  it 
originated  in  the  presence  of  other  distracting  sounds. 
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Part  2  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

29.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

30.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

3 1 .  Are  there  abilities  required  of  first-term  Soldiers  in  your  MOS  that  were  not  reflected  in  the 
questionnaire?  If  so,  what? 

<SPACE> 

32.  Please  enter  any  other  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Army  Work  Styles  Questionnaire 


Instructions 

The  purpose  of  this  questionnaire  is  to  find  out  about  the  types  of  work  styles  required  of  first- 
term  Soldiers  in  your  MOS.  Work  styles  are  personal  characteristics  that  can  affect  how  well 
someone  performs  a  job.  The  Army  offers  many  different  MOS;  your  answers  will  let  us  know 
the  work  styles  that  are  most  and  least  important  to  yours.  As  you  complete  this  questionnaire, 
consider  the  typical  first-term  Soldier  in  your  MOS  who  is  at  Skill  Level  1  (SL1)  and  who  has 
completed  at  least  one  year  of  duty  in  a  typical  unit  assignment  and  position  and  at  least  one 
series  of  Green-Amber-Red  training  cycles  and  unit  operational  requirements. 

Rate  each  work  style  in  terms  of  its  importance  to  the  job  of first-term  Soldiers  in  your  MOS. 
You  will  use  the  following  scale  to  make  this  rating: 


1 

2 

3 

4 

5 

Not  Important 

Somewhat 

Important 

Important 

Very  Important 

Extremely 

Important 

When  completing  this  questionnaire,  it  is  important  that  you  consider  how  well  each  work  style 
describes  the  job  of first-term  Soldiers  in  your  MOS,  not,  the  job  of  NCOs  in  your  MOS. 
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Part  1  -  Work  Styles 

First-term  Soldiers  in  my  MOS  perform  work  that  requires. . . 


# 

Work  Style 

Importance 

d-5) 

1 

Showing  a  cooperative  and  friendly  attitude  towards  others  they 
dislike  or  disagree  with. 

2 

Being  open  to  change  (positive  or  negative)  and  a  lot  of  variety. 

3 

Leading,  taking  charge,  and  giving  direction. 

4 

Accomplishing  tasks  alone,  with  little  supervision  or  help  from 
others. 

5 

Setting  challenging  goals  and  working  continuously  to  attain 
them. 

6 

Consistently  meeting  obligations  and  completing  duties  on  time. 

7 

Dealing  effectively  with  high-stress  situations  and  accepting 
frequent  criticism. 

8 

Much  creativity  and  original  thinking  to  perform  successfully. 

9 

Maintaining  composure  and  keeping  emotions  and  behavior  in 
check  even  in  very  difficult  circumstances. 

10 

Being  sensitive  to  others'  needs  and  feelings  and  being 
understanding. 

11 

High  levels  of  energy  and  stamina  to  perform  successfully. 

12 

Working  closely  with  others  (instead  of  alone)  to  get  tasks 
completed. 

13 

Being  thorough  and  paying  close  attention  to  details. 

14 

Perfonning  tasks  that  take  a  long  time  to  "get  right"  and 
overcoming  several  obstacles  along  the  way. 

15 

Taking  on  new  or  additional  responsibilities  that  may  fall  outside 
of  their  job  duties. 

16 

Interacting  with  people  of  different  cultures  and  backgrounds,  and 
appreciating  differences  in  their  values,  opinions,  and  beliefs. 

17 

Being  organized  and  efficient. 

18 

Thinking  through  things  logically. 

19 

Following  rules,  procedures,  and  protocols,  even  when  those 
requirements  are  personally  inconvenient. 

20 

Being  decisive  and  taking  immediate  action  in  response  to  the 
situation. 
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Part  2  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

33.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

34.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

35.  Are  there  work  styles  required  of  first-term  Soldiers  in  your  MOS  that  were  not  reflected  in 
the  current  questionnaire?  If  so,  what? 

<SPACE> 

36.  Please  enter  any  other  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Army  Work  Interests  Questionnaire 


Instructions 

The  purpose  of  this  survey  is  to  find  out  about  the  work  interests  supported  by  the  kinds  of  work 
performed  by  first-term  Soldiers  in  your  MOS.  Work  interests  are  preferences  for  certain  kinds  of 
work  that  can  influence  an  individual’s  satisfaction  with  their  job.  The  Anny  offers  many 
different  MOS;  your  answers  will  let  us  know  the  work  styles  that  are  most  and  least  important 
to  yours.  As  you  complete  this  questionnaire,  consider  the  typical  first- term  Soldier  in  your  MOS 
who  is  at  Skill  Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a  typical  unit 
assignment  and  position  and  at  least  one  series  of  Green- Amber-Red  training  cycles  and  unit 
operational  requirements. 

Rate  each  work  interest  in  terms  of  its  importance  to  the  job  of  first-term  Soldiers  in  your  MOS. 
You  will  use  the  following  scale  to  make  this  rating: 


1 

2 

3 

4 

5 

Not  Important 

Somewhat 

Important 

Important 

Very  Important 

Extremely 

Important 

When  completing  this  questionnaire,  it  is  important  that  you  consider  how  well  each  work 
interest  describes  the  job  of first-term  Soldiers  in  your  MOS,  not,  the  job  of  NCOs  in  your  MOS. 
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Part  1  -  Work  Interests 

First-term  Soldiers  in  my  MOS  perform  must  like. . . 

Importance 

_ #  Work  Interest _ (1-5) 

1  Communicating  with  others  (verbally  or  in  writing)  inside  or  outside 

_ their  organization. _ 

2  Protecting  others. 

3  Assisting  and  caring  for  others. 

4  Selling  and  influencing  others. 

5  Resolving  conflicts  and  negotiating  with  others. 

6  Performing  for  or  working  directly  with  the  public. 

7  Training  and  teaching  others. 

8  Entering,  transcribing,  or  recording  data  or  information  in  written  or 

_ electronic  form. _ 

9  Processing  or  tabulating  data  or  compiling  information  (visual, 

_ audio,  verbal,  or  written)  and  verifying  its  accuracy  or  completeness. _ 

10  Translating  or  explaining  what  data  or  information  (visual,  audio, 

_ verbal,  or  written)  means  to  others. _ 

1 1  Analyzing  data  or  interpreting  information  (visual,  audio,  verbal,  or 

_ written). _ 

12  Observing  or  collecting  data  or  information  from  multiple  sources  or 

_ media  (visual,  audio,  print,  electronic). _ 

13  Identifying  objects,  actions,  and  events. 

14  Monitoring  processes,  materials,  or  surroundings. 

1 5  Inspecting  equipment,  structures,  or  materials  for  problems  or  defects. 

16  Estimating  the  quantifiable  characteristics  of  products,  events,  or 
information. 

1 7  Judging  the  quality  of  objects,  services,  or  people. 

18  Evaluating  information  to  determine  compliance  with  standards. 

19  Scheduling  and  coordinating  events,  programs,  or  activities. 

20  Drafting  or  laying  out  specifications  for  technical  devices,  parts,  or 

_ equipment. _ 

21  Performing  physical  activities,  such  as  running,  walking,  climbing, 
lifting,  or  balancing. 

22  Handling  and  moving  objects  or  materials. 

23  Shooting,  firing,  and  operating  weapons  or  weapon  systems  (small 

_ arms,  missile  systems,  and  other  munitions). _ 

24  Working  with  computers  or  computer  networks. 

25  Driving  or  operating  vehicles  or  other  mechanized  equipment. 

26  Repairing  and  maintaining  mechanical  equipment. 

27  Repairing  and  maintaining  electronic  or  electrical  equipment. 

28  Constructing  buildings,  bridges,  walls,  or  structures. 

29  Performing  administrative  activities,  such  as  handling  paperwork 

_ and  completing  forms. _ 

30  Monitoring  and  controlling  resources. 
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Part  2  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

37.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

38.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

39.  Are  there  work  interests  supported  by  the  kinds  of  work  perfonned  by  first-term  Soldiers  in 
your  MOS  that  were  not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

40.  Please  enter  any  other  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Army  Work  Values  Questionnaire 


Instructions 

The  purpose  of  this  questionnaire  is  to  find  out  about  the  work  values  supported  by  the  kinds  of 
work  performed  by  first-term  Soldiers  in  your  MOS.  Work  values  are  personal  characteristics  that 
can  influence  how  satisfied  an  individual  is  in  their  job.  The  Army  offers  many  different  MOS; 
your  answers  will  let  us  know  the  work  values  that  are  most  and  least  important  to  yours.  As  you 
complete  this  questionnaire,  consider  the  typical  first-term  Soldier  in  your  MOS  who  is  at  Skill 
Level  1  (SL1)  and  who  has  completed  at  least  one  year  of  duty  in  a  typical  unit  assignment  and 
position  and  at  least  one  series  of  Green-Amber-Red  training  cycles  and  unit  operational 
requirements. 

Rate  each  work  value  in  terms  of  its  importance  to  the  job  of  first-term  Soldiers  in  your  MOS. 
You  will  use  the  following  scale  to  make  this  rating: 


1 

2 

3 

4 

5 

Not  Important 

Somewhat 

Important 

Important 

Ven>  Important 

Extremely 

Important 

When  completing  this  questionnaire,  it  is  important  that  you  consider  how  well  each  work  value 
describes  the  job  of first-term  Soldiers  in  your  MOS,  not,  the  job  of  NCOs  in  your  MOS. 
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Part  1  -  Work  Values 


First-term  Soldiers  in  my  MOS. . . 

Importance 

_ #  Item _ (1-5) 

1  Do  work  that  most  people  admire  and  respect 

2  Have  opportunities  for  advancement 

3  Plan  their  work  with  little  supervision 

4  Have  supervisors  who  provide  a  lot  of  support  and  guidance 

5  Have  the  freedom  to  pursue  their  non-work  interests  after  hours 

6  Work  in  a  comfortable  and  relaxed  environment 

7  Get  a  feeling  of  accomplishment 

8  Do  work  that  makes  a  valuable  contribution  to  society 

9  Do  their  work  alone 

1 0  Help  others 

1 1  Have  clear-cut  duties  and  responsibilities 

12  Have  something  different  to  do  every  day 

1 3  Have  opportunities  to  lead  others 

14  Get  feedback  about  my  performance 

15  Travel  and  often  be  far  from  home 

1 6  Have  the  opportunity  to  improve  their  physical  fitness 

1 7  Do  something  that  makes  use  of  their  abilities 

18  Try  out  their  own  ideas 

1 9  Receive  recognition  for  what  they  do 

20  Develop  friendships  with  their  co-workers 

2 1  Always  have  enough  work  to  keep  themselves  busy 

22  Have  a  flexible  work  schedule 

23  Learn  new  skills 

24  Establish  roots  in  a  community  by  not  having  to  move  frequently 

25  Feel  valued  by  their  organization 

26  Gain  personal  discipline  and  maturity 

27  Make  decisions  that  affect  the  work  of  others 

28  Work  closely  with  others 

29  Solve  difficult  and  complex  problems 

30  Do  work  that  is  high  in  prestige  and  status 
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Part  2  -  Feedback 

Please  take  a  moment  to  reflect  back  on  the  questionnaire  you  completed  and  answer  the 
following  questions: 

41.  How  would  you  rate  the  questionnaire  on  the  following: 
a.  Ease  of  use: 


Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

b.  Instructions: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

c.  Clarity  of  the  statements  you  were  asked  to  rate: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

d.  Relevance  to  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

e.  Usefulness  for  describing  the  requirements  of  your  MOS: 

Very  Bad 

Bad 

Neither  Bad 

Nor  Good 

Good 

Very  Good 

1 

2 

3 

4 

5 

42.  If  you  rated  the  questionnaire  a  3  or  below  (“Neither  Bad  Nor  Good”  to  “Very  Bad”)  on  any 
of  the  above  questions,  please  explain. 

<SPACE> 

43.  Are  there  work  values  supported  by  the  kinds  of  work  perfonned  by  first-term  Soldiers  in 
your  MOS  that  were  not  reflected  in  the  current  questionnaire?  If  so,  what? 

<SPACE> 

44.  Please  enter  any  other  feedback  or  comments  on  the  questionnaire  in  the  space  below: 

<SPACE> 
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Appendix  F 


Cross-Job  Work  Activity  Descriptors  and  Definitions 


F-l 


Major  Duty 


Definition 


1  Protect  against  NBC  hazards  Uses  protective  clothing,  masks,  and  decontamination  equipment  to  protect 

self,  others,  equipment,  and  supplies  from  nuclear,  biological,  and 
chemical  (NBC)  hazards.  Detects  and  monitors  potential  hazards  using 
NBC  detection  equipment. 

2  Handle  demolitions  or  mines  Stores,  places,  charges,  discharges,  and  disarms  explosives,  demolition 

devices,  or  mines. 

3  Engage  in  hand-to-hand  combat  Uses  offensive  and  defensive  maneuvers  to  combat  and  protect  self  and 

others  from  hostile  individuals. 

4  Inspect  and  maintain  weapons  Checks,  disassembles,  assembles,  cleans,  lubricates,  and  adjusts  weapons, 

including  pistols,  rifles,  machine  guns,  hand  grenades,  and  breechblocks. 

5  Fire  direct  fire  weapons  Aims,  tracks,  and  fires  individual  weapons  (e.g.,  rifles,  pistols,  machine 

guns,  hand  grenades)  at  designated  targets.  Prepares  and  loads  weapon. 
Clears  and  unloads  weapon.  Arms  and  throws  hand  grenades. 

6  Troubleshoot  and  repair  weapons  Finds  the  cause  of  malfunctions  in  weapons  using  technical  manuals,  tools, 

and  test  equipment.  After  the  cause  of  a  problem  in  a  weapon  has  been 
found,  fixes  it  using  the  appropriate  tools  and  necessary  replacement  parts 
by  following  directions  in  the  weapon's  technical  manual. 

7  Navigate  from  point  to  point  Reads  and  interprets  maps,  other  navigational  devices,  and  equipment 

(e.g.,  GPS)  to  locate  position  of  self  and  others.  Determines  grid 
coordinates  and  directions.  Moves  from  point  to  point  in  response  to 
terrain  features  (e.g.,  or  cover  or  concealment)  battle  conditions,  and 
mission,  with  or  without  the  aid  of  maps  and  other  navigational  equipment. 

8  Maintain  personal  and  Selects,  prepares,  and  occupies  individual  tactical  positions  (e.g.,  battle 

operational  security  positions,  overwatch  positions,  observations  posts),  camouflages  self  and 

equipment,  and  observes  security  procedures. 

9  Provide  emergency  first  aid  Provide  emergency  first  aid  to  individuals  (e.g.,  CPR,  put  on  field  dressing, 

prevent  shock). 

10  Scout  and  identify  targets  With  or  without  optical  devices  and  other  equipment  (e.g.,  night  sights, 

weapon  sights,  binoculars),  scouts  and  locates  possible  targets  and  their 
position(s).  Identifies  target's  type  (e.g.,  troops,  tanks,  aircraft)  and  intent 
(e.g.,  hostile  or  non-hostile). 

1 1  Fire  heavy  direct  fire  weapons  Aims,  tracks,  and  fires  heavy  direct  fire  weapon  (e.g.,  tank  main  guns, 

TOW  missile)  at  targets.  Positions  and  loads  weapons  for  firing.  Unloads 
or  extracts  unused  rounds  or  misfires. 

12  Fire  indirect  fire  weapons  Aims,  tracks,  and  fires  indirect  weapon  (e.g.,  field  artillery,  heavy  mortars)  by 

adjusting  azimuth  and  elevation  controls  to  hit  designated  targets.  Positions 
and  loads  weapon  for  firing.  Unloads  or  extracts  unused  rounds  or  misfires. 

13  Drive  track  vehicles  Drives  track  vehicles  (e.g.,  tank,  APC,  BFV,  etc.)  in  response  to  mission, 

terrain,  and  traffic  controls. 

14  Install  and  maintain  electronic  Installs  and  connects  electronic,  communications,  and  satellite-based 

equipment  equipment  (e.g.,  GPS,  radios,  antennas,  satellite  telephones,  radar,  missile 

and  tank  ballistics  computer  systems).  Inspects  and  monitors  equipment  for 
operation.  Conducts  scheduled  services  to  maintain  equipment.  (Does  not 
include  personal  computers  and  computer  networks.) 
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15  Troubleshoot  and  repair 
electronic  equipment 


16  Install  and  maintain  electrical 
and  power  transmission  systems 


17  Troubleshoot  and  repair 

electrical  and  power  transmission 
systems 


18  Install  and  maintain  personal 
computers  and  peripheral 
equipment 

19  Install  and  maintain  computer 
networks 

20  Troubleshoot  and  repair  personal 
computers  and  computer 
networks 


21  Operate  electronic  equipment 


22  Operate  personal  computers  and 
networks 

23  Record  and  document 
audiovisual  information 

24  Send  and  receive  radio  messages 

25  Collect  and  decode  electronic 
signals 


Troubleshoots  electronic,  communications,  and  satellite-based  equipment 
(e.g.,  GPS,  radios,  antennas,  radar,  missile  and  tank  ballistics  computer 
systems)  to  diagnose  problems  and  malfunctions  using  specialized  test 
equipment  and  manuals.  Repairs  equipment  with  the  appropriate  tools 
(e.g.,  test  sets,  screwdrivers,  pliers,  soldering  guns)  and  necessary 
replacement  parts  by  following  directions  in  the  equipment's  technical 
manual.  (Does  not  include  personal  computers  and  computer  networks.) 

Installs  and  connects  electrical  and  power  transmission  systems  (e.g., 
electrical  wiring,  power  cables,  communications  wiring).  Lays,  splices, 
and  knots  wires  and  cables  using  the  appropriate  tools  (e.g.,  wire  cutters). 
Inspects  and  monitors  systems  for  operation.  Conducts  scheduled  services 
to  maintain  systems. 

Measures  and  tests  electrical  and  power  transmission  system  components  (e.g., 
generators,  wiring  harnesses,  switches,  relays,  circuit  breakers,  wires,  cables) 
to  detect  and  diagnose  problems  and  malfunctions  using  specialized  test 
equipment  and  manuals.  Repairs  system  components  with  the  appropriate  tools 
(e.g.,  wire  strippers,  pliers,  soldering  irons)  and  necessary  replacement  parts  by 
following  directions  in  the  equipment's  technical  manual. 

Connects  personal  computers  and  peripherals,  installs  software,  and 
connects  to  networks.  Monitors  computers  and  peripherals.  Conducts 
scheduled  services  and  upgrades  to  maintain  computers  and  peripherals. 
(Does  not  include  computer  networks.) 

Installs  and  configures  network  hardware  and  software.  Monitors  network 
use  and  performance.  Conducts  scheduled  services  and  upgrades  to 
maintain  network.  (Does  not  include  personal  computers.) 

Troubleshoots  personal  computer  and  network  components  (e.g.,  hard  drive, 
monitors,  keyboard,  network  router,  network  cables-wiring)  to  detect  and 
diagnose  problems  and  malfunctions  using  specialized  test  equipment  and 
manuals.  Repairs  computer  or  network  components  with  the  appropriate  tools 
(e.g.,  pliers,  screwdrivers,  wrenches)  and  necessary  replacement  parts  by 
following  directions  in  the  equipment's  technical  manual. 

Sets  and  adjusts  the  controls  to  operate  electronic,  communications,  and 
satellite-based  equipment  (e.g.,  GPS,  radios,  antennas,  satellite  telephones, 
radar,  missile  ballistics  controls),  including  tactical  command  and  control 
systems  (e.g.,  Force  XXI  Battle  Command,  Brigade-and-Below  [FBCB2]). 

Operates  and  works  with  personal  computers  and  networks  to  create  and 
edit  documents  and  presentations,  store  and  enter  data  into  databases,  or  to 
search  for  and  process  information.  (Does  not  include  programming). 

Records  and  documents  visual  and  sound  information  for  intelligence 
analysis,  training,  or  documentation  using  audiovisual  equipment  (e.g., 
audio  recorders,  cameras,  videotape,  digital  video). 

Uses  standardized  radio  codes  and  procedures  to  transmit  and  receive 
messages  and  other  information. 

Collects  electronic  signals  and  communications.  Uses  coding  systems  and 
rules  to  decipher  and  interpret  coded  information. 
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26  Analyze  electronic  signals  Analyzes  electronic  signals  to  detect  threat  transmitters  and  electronic 

countermeasures. 

27  Provide  data  processing  and  Analyzes  data  processing  needs.  Selects  or  prepares,  edits,  tests,  and  runs 

programming  support  computer  programs.  Documents  process  and  results. 

28  Produce  maps,  overlays,  or  range  Uses  drafting,  graphics,  and  related  techniques  to  draw  and  revise  maps  of 

cards  terrain,  including  locations  of  buildings  and  other  objects,  targets,  avenues 

of  approach,  and  maneuver  areas  from  personal  observation  or  available 
materials  (e.g.,  aerial  photographs). 

29  Provide  technical  guidance  and  Explains  and  demonstrates  to  Army  and  non- Army  personnel  at  all  levels 

advice  on  the  installation,  how  to  install,  maintain,  and  use  equipment.  Answers  technical  questions, 

maintenance,  and  use  of  Provides  expert  advice  to  others  on  issues  related  to  the  installation, 

equipment  maintenance,  and  use  of  equipment. 

30  Translate  foreign  languages  Translates  written  or  spoken  foreign  language  communications. 

31  Analyze  intelligence  data  Determines  importance  and  reliability  of  information.  Uses  information  to 

determine  identity,  capabilities,  disposition,  and  movement  of  non-U.  S. 
forces  and  personnel. 

32  Control  individuals  and  crowds  Performs  guard  duty,  including  challenge  and  password.  Apprehends  and 

searches  suspected  criminals,  detainees,  or  prisoners.  Guards  and  escorts 
detainees  or  prisoners.  Participates  in  riot  control. 

33  Collect  information  from  and  on  Collects  and  gathers  information  from  and  on  individuals  and  groups  using 

individuals  and  groups  a  variety  of  techniques  (e.g.,  interviews,  focus  groups,  observations). 

34  Inspect  and  maintain  mechanical  Inspects  and  monitors  mechanical  equipment  (e.g.,  vehicles,  trailers, 

equipment  generators,  construction  equipment).  Conducts  scheduled  services  to 

maintain  equipment. 

Troubleshoots  mechanical  system  components  (e.g.,  engines, 
transmissions,  brakes,  hydraulics,  refrigeration  systems)  to  diagnose 
problems  and  malfunctions  using  specialized  test  equipment  and  manuals. 
Repairs  equipment  using  the  appropriate  tools  (e.g.,  wrenches, 
screwdrivers,  gauges,  hammers,  soldering  equipment)  and  necessary 
replacement  parts  by  following  directions  in  the  equipment  manual. 

36  Operate  gas  and  electric  powered  Operates  gas  and  electric  powered  equipment  (e.g.,  electric  generators,  air 

equipment  compressors,  smoke  generators,  quarry  machines,  mobile  washing 

machines,  water  pumps)  to  produce  power  or  process  materials. 

37  Prepare  and  process  forms  Follows  standardized  procedures  to  prepare  or  complete  forms  and 

documents  (e.g.,  personnel  records,  legal  briefs,  requisition  requests, 
inspection  records).  Obtains  required  authorizations,  as  needed.  Monitors 
and  reviews  forms  for  completeness. 

38  Maintain  records  and  files  Collects,  sorts,  indexes,  files,  and  retrieves  records  and  files  (e.g.,  medical 

records,  training  rosters,  personnel  statistics,  supply  inventories,  etc.). 

39  Write  documents  and  Prepares  and  writes  letters,  reports,  and  memos.  Proofreads  and  edits 

correspondence  documents  prior  to  distribution. 

40  Monitor  and  control  financial  Monitor  and  controls  the  expenditure  of  financial  resources.  Maintains  and 

resources  reviews  accounting  records.  Disperses  and  collects  money  and  money  orders. 


35  Troubleshoot  and  repair 
mechanical  equipment 
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41  Load  and  unload  supplies  Builds  or  assembles  platforms,  cushions,  and  riggings  for  supplies  and 

equipment  to  protect  from  damage  during  transport.  Loads  and  lashes 
materials  onto  transport  vehicles  (land,  sea,  or  air)  to  secure  and  protect 
from  damage  or  loss  during  shipment.  Unpacks  and  unloads  supplies  after 
transport  to  designated  location. 

42  Drive  wheeled  vehicles  Drives  wheeled  vehicles  over  roads  and  cross-country  in  response  to 

mission,  terrain,  and  traffic  regulations. 

43  Drive  water  craft  Drives  water  craft  (e.g.,  boats,  rafts)  in  response  to  mission  requirements 

and  nautical  regulations. 

44  Inspect,  store,  and  issue  supplies  Inspects  supplies  and  reviews  paperwork  upon  receipt.  Sorts  and  stores 

supplies.  Issues  or  ships  supplies  to  authorized  personnel  or  units. 

45  Order  supplies  and  equipment  Determines  supply  and  equipment  needs  or  evaluates  requests.  Prepares 

and  submits  orders  and  requisitions  for  needed  supplies  and  equipment. 

46  Manage  and  control  traffic  Manages  and  coordinates  the  departing,  en  route,  arriving,  and  holding  of 

traffic  (land,  air,  or  sea)  by  monitoring  equipment,  communicating  with 
vehicles  and  other  traffic  control  units. 

47  Operate  hand-operated  power  Uses  hand-operated  power  excavating  equipment  (e.g.,  air  hammers  and 

excavating  equipment  drills,  paving  breakers,  grinders,  backfill  tampers)  to  build  concrete,  stone, 

or  other  structures. 

48  Operate  heavy  equipment  Operates  heavy  equipment  (e.g.,  fork  lifts,  cranes,  back-hoes,  and  graders) 

to  load,  unload,  or  move  other  heavy  equipment,  supplies,  construction 
materials  (e.g.,  culvert  pipe,  building  and  bridge  parts)  or  terrain  (e.g., 
earth,  rocks,  trees,  etc.). 

49  Install,  maintain,  and  repair  Installs  plastic  or  fiberglass  parts  and  structures.  Fixes  plastic  or  fiberglass 

plastic  and  fiberglass  parts  and  structures  by  cutting,  sawing,  drilling,  sanding,  filling,  gluing, 

and  painting. 

50  Repair  metal  structures  or  parts  Fixes  metal  structures  or  parts  by  bending,  cutting,  drilling,  welding, 

hammering,  grinding,  soldering,  and  painting. 

51  Construct  metal  or  steel  Erects  bridges,  communication  antennas,  and  other  steel  structures.  May 

structures  require  the  assistance  of  others  and  use  of  heavy  equipment. 

52  Install,  maintain,  and  repair  pipe  Installs,  connects,  and  tests  pipe  assemblies  and  fixtures  (e.g.,  plumbing, 

assemblies  POL  pipelines  and  pumps).  Repairs  pipe  assemblies. 

53  Construct  wooden  buildings  and  Measures,  saws,  nails  or  planes  to  frame,  sheathe,  and  roof  buildings,  or 

structures  erects  trestles,  bridges,  and  piers  from  wood. 

54  Construct  masonry  buildings  and  Measures,  lays  brick  or  concrete  blocks,  or  builds  forms  and  pours 

structures  concrete  to  construct  walls,  columns,  field  fortifications,  and  other 

concrete  or  masonry  structures. 

55  Produce  technical  drawings  and  Uses  drafting  equipment  or  other  media  (e.g.,  pen,  pencil,  paint)  to  make 

illustrations  technical  drawings  and  blueprints.  (Does  not  include  maps,  range  cards,  or 

other  field  expedient  drawings.) 

56  Prepare  food  and  beverages  Prepares  food  and  beverages  according  to  recipes  and  meal  plans  (measure, 

mix,  bake).  Inspects  fresh  food  and  staples  for  freshness.  Cleans  equipment 
and  work  area. 
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57  Prepare  patients  and  equipment 
for  medical  procedures 

58  Provide  medical  treatment 


59  Schedule  patients  and  medical 
services 

60  Provide  counseling  and  other 
interpersonal  interventions 

61  Perform  laboratory  procedures 

62  Collect  and  analyze  weather  and 
environmental  data 

63  Conduct  land  surveys 

64  Deliver  presentations 

65  Reproduce  printed  materials 

66  Demonstrate  military  presence 


67  Maintain  physical  fitness 


68  Manage  own  duties  and 
responsibilities 


69  Demonstrate  extra  effort  and 
personal  initiative  on  the  job 


70  Manage  own  professional 
development 


Prepares  patients  for  medical  procedures  by  following  prescribed  rules  and 
directions.  Prepares  medical  or  dental  treatment  areas  for  use  by  laying  out 
instillments  and  equipment.  Cleans  equipment  and  area  for  future  use. 

Provides  medical  treatment  to  Soldiers  in  the  field  or  in  medical  or  dental 
clinics,  or  administers  veterinary  treatment  to  animals  (e.g.,  administers 
injections,  takes  blood  pressure,  changes  sterile  dressings).  (Does  not 
include  performing  basic  first  aid.) 

Schedules  and  provides  routine  information  to  persons  seeking  medical, 
dental,  or  counseling  services. 

Counsels  individuals  and  groups  (e.g.,  families)  on  personal  issues  and 
relationship  problems  in  a  clinical,  non-supervisory  setting.  (Does  not 
include  coaching  and  counseling  Soldiers  on  performance-based  issues.) 

Prepares  and  handles  samples  for  laboratory  tests  (e.g.,  medical,  chemical, 
biological).  Conducts  various  types  of  laboratory  tests  following 
prescribed  protocols  and  procedures.  Files  and  reports  results. 

Collects  data  and  information  on  weather  and  environmental  conditions. 
Analyzes  their  effects  on  tactical  operations. 

Surveys  terrain  to  determine  elevations,  azimuths,  and  distances  of  terrain 
features.  Records  information. 

Makes  formal  presentations  (e.g.,  briefings,  radio  and  television  broadcasts). 

Reproduces  printed  materials  using  duplicating  machines  (e.g.,  electronic 
copiers,  printing  presses).  Collates  and  binds  materials  using  various  types 
of  bindery  equipment. 

Presents  a  positive  and  professional  image  of  self  and  the  Army  even  when 
off  duty.  Maintains  proper  military  appearance.  Sets  the  precedent  for 
other  Soldiers  to  follow. 

Meets  Army  standards  for  weight,  physical  fitness,  and  strength.  Maintains 
health  (e.g.,  dental  hygiene)  and  fitness  to  meet  requirements,  to  handle  the 
physical  demands  of  the  daily  job,  and  to  endure  the  stress  of  combat. 

Manages  own  responsibilities  (e.g.,  work  assignments,  personal  finances, 
family,  and  personal  well  being),  and  appears  on  duty  prepared  for  work. 
Sets  goals,  makes  plans,  and  critically  evaluates  own  performance.  Works 
effectively  without  direct  supervision  but  seeks  help  when  appropriate. 

Demonstrates  high  effort  in  completing  work.  Takes  independent  action 
when  necessary.  Seeks  out  and  willingly  accepts  responsibility  and 
additional  challenging  assignments.  Persists  in  carrying  out  difficult 
assignments  and  responsibilities. 

Develops  job-related  skills,  devoting  time  off-duty  to  study  and  practice 
important  job-related  skills.  Takes  on  additional  job  duties  and 
responsibilities  to  prepare  for  promotion  and  actively  seeks  out 
opportunities  for  self-improvement.  Keeps  up-to-date  technically. 
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71 


72 


73 


74 


75 


76 


77 


78 


79 


80 


81 


Demonstrate  personal  integrity 


Exhibit  self-control 


Follow  orders  and  rules 


Contribute  to  team  tasks 


Direct  peers  and  individual  team 
members 


Support  peers  and  individual 
team  members 


Train  peers  and  individual  team 
members 


Flelp  peers  and  individual  team 
members 


Monitor  peer  and  individual  team 
member  performance 


Monitor  team  performance 


Contribute  to  team  planning 


Maintains  high  ethical  standards.  Does  not  succumb  to  peer  pressure  to 
commit  prohibited,  harmful,  or  questionable  acts.  Voluntarily  reports  thefts, 
misconduct,  and  any  other  violations  of  military  order  and  discipline. 
Understands  and  accepts  the  basic  values  of  the  Army  and  acts  accordingly. 

Controls  personal  behavior.  Does  not  engage  in  negative  behaviors,  such 
as  alcohol  and  substance  abuse  at  work.  Meets  financial  obligations 
consistently,  displays  emotional  maturity,  and  does  not  allow  personal 
matters  to  interfere  with  professional  duties  and  obligations. 

Understands  and  carries  out  orders  relayed  orally  or  in  writing.  Displays 
respect  for  authority.  Adheres  to  regulations,  policies,  and  procedures 
while  completing  assignments.  Checks  the  behavior  of  others  to  ensure 
compliance. 

Takes  ownership  for  and  completes  assigned  tasks  for  team  according  to 
committed  timelines.  Demonstrates  effort  toward  team  goals.  Does  not 
pass  work  off  to  others  or  take  shortcuts  that  compromise  quality. 

Flelps  to  define  goals  and  organize  and  prioritize  tasks  for  peers  and  individual 
team  members.  Generates  plans  and  strategies  for  task  completion,  identifies 
resources  needed  to  meet  team  goals,  and  shares  resources  or  guides  individual 
team  members  to  resources  to  help  complete  their  tasks. 

Provides  social  support  and  empathy,  offers  verbal  encouragement  and  acts 
respectfully  towards  peers  and  team  members,  especially  when  tasks  or 
situations  are  difficult  or  demanding.  Facilitates  cohesion  and  effective 
working  relationships  between  team  members  by  acting  honestly, 
communicating  openly  and  helping  to  manage  or  resolve  conflicts.  Does  not 
embarrass  team  members  in  front  of  others,  act  impatiently,  or  blame  others. 

Shares  information  with  peers  and  individual  team  members,  provides  task 
explanations  and  demonstrations,  answers  questions,  and  gives  timely  and 
constructive  feedback.  Does  not  withhold  information  about  team-related 
tasks. 

Fills  in  or  covers  for  peers  or  individual  team  members  who  are 
overwhelmed  or  absent.  Rearranges  own  schedule  and  demonstrates 
flexibility  to  help  other  peers  or  individual  team  members.  Puts  in  extra 
time  and  effort  to  help  peers  and  team  members  without  being  asked  and 
without  complaining.  Does  not  engage  in  off-task  activities  when  other 
team  members  could  use  help. 

Observes  and  is  knowledgeable  about  the  performance  of  peers  or  other 
team  members.  Pays  attention  to  what  peers  and  individual  team  members 
are  doing.  Evaluates  progress  of  self  and  others  and  recognizes  when  team 
members  may  need  help. 

Pays  attention  to  the  team's  situation,  including  relevant  conditions, 
procedures,  policies,  resources,  systems,  equipment,  technology,  and  level 
of  team  accomplishment.  Notices  and  identifies  team-relevant  problems 
and  obstacles. 

Flelps  in  identifying  alternative  solutions,  strategies,  or  options  for  dealing 
with  problems,  obstacles,  or  decisions.  Flelps  in  evaluating  alternative 
courses  of  action,  and  takes  preventive  measures  to  avoid  future  problems. 
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82  Contribute  to  team  coordination  Contributes  to  and  encourages  discussion  of  work  distribution,  workload 

balance,  potential  workload  problems,  and  the  sequencing  of  team  members 
activities.  Coordinates  own  task  activities  with  other  team  members.  Does  not 
make  unnecessary  requests  or  overload  other  team  members. 

Plans  major  operations  or  team  tasks  prior  to  their  actual  execution  in  field 
or  workplace.  Translates  goals  and  objectives  into  tasks  and  activities. 
Forecasts  possible  problems  for  the  platoon/squad/team  and  develops 
strategies  for  addressing  these  problems.  Organizes  and  prioritizes  work. 

Provides  guidance  and  direction  to  individual  Soldiers.  Motivates  Soldiers 
by  providing  them  with  recognition,  encouragement,  constructive 
criticism,  and  other  feedback  as  appropriate.  Helps  to  set  goals  and 
maintains  performance  standards  for  Soldiers.  Monitors  and  counsels 
Soldiers  on  specific  performance  or  personal  problems,  as  needed. 

85  Train  and  coach  Soldiers  Trains,  instincts,  and  coaches  Soldiers  on  how  to  complete  technical  job 

tasks.  Assists  Soldiers  in  improving  their  technical  job  skills/proficiency. 

86  Communicate  information  to  Keeps  Soldiers,  superiors,  and  others  informed  about  factors  and  issues 
Soldiers,  peers,  and  superiors  affecting  them.  Obtains  and  then  passes  on  information  to  those  who 

should  know. 

87  Administer  personnel  actions  and  Completes  performance  appraisals.  Makes  or  recommends  various 

procedures  personnel  actions.  Keeps  and  maintains  adequate  records.  Follows  standard 

operating  procedures. 

88  Manage  and  monitor  Keeps  an  operation  going  once  it  has  been  initiated.  Checks  to  make  sure  that 

operations/missions  and  team  Soldiers  are  carrying  out  then  duties  properly.  Makes  sure  they  have  the  right 
tasks  equipment.  Monitors  or  evaluates  the  status  of  equipment  readiness. 

89  Direct  and  lead  Directs  and  leads  platoon/squad/team  activities.  Assigns  NCOs  and  Soldiers 

platoons/squads/teams  duties  and  responsibilities  for  completing  platoon/squad/team  tasks. 

Coordinates  the  actions  of  squads/teams  within  unit  and  those  of  individual 
Soldiers. 

90  Model  correct  behavior  to  Models  the  correct  performance  behavior  to  Soldiers,  whether  it  be 

Soldiers  technical  task  performance  under  adverse  conditions  or  exhibiting 

appropriate  military  bearing. 

91  Support  individual  Soldiers  Demonstrates  personal  concern  for  Soldiers.  Backs  up  and  supports 

Soldiers  as  appropriate.  Looks  out  for  their  welfare. 

92  Build  and  manage  Builds  and  manages  platoon/squad/team  cohesion.  Manages  and  resolves 

platoon/squad/team  cohesion  internal  conflict  among  team  members.  Promotes  and  sustains  team  morale. 

93  Engage  and  negotiate  with  host  Interacts  and  meets  with  host  nationals  and  local  leaders  (e.g.,  tribal,  police 

nationals  and  local  leaders.  chiefs)  to  obtain  information,  handle  complaints,  settle  disputes,  and 

resolve  grievances  between  and  among  host  nationals.  Negotiates  with  host 
nationals  to  resolve  conflicts  with  U.S.  forces  and  to  secure  their  support 
for  U.S.  military  operations. 

94  Coordinate  with  other  units  and  Coordinates  with  other  units  and  non- Army  personnel  (e.g.,  contractors, 

non- Army  personnel  host  nation  forces)  before,  during,  and  after  operations  for  support  and 

equipment.  Shares  information  on  status,  position,  and  actions  with  other 
units  and  non- Army  personnel,  as  needed. 


83  Plan  and  organize 
operations/missions  and  team 
tasks 

84  Direct  and  motivate  individual 
Soldiers 
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